Guillotina uses a PostgreSQL-compatible database as the source of truth and Elasticsearch indexing of PostgreSQL JSONB. This project involves creating a new component that handles the storage and indexing in a new, distributed piece of software: Guillotina Object Server. The main values of this project are transactional support, distribution of information, and indexing technology using a low-level language: Rust.
The aim is to build a small and fast component that handles transactions and keyword/full-text searches from Guillotina. There is already a basic implementation which shows the overall design of the concept with a Protobuffer protocol, Rust bindings to RocksDB and basic integration with Tantivy (full-text indexer). Based on top of Tokio async framework on Rust and Protobuffer protocol this project has four goals:
Storing python pickles on a Rocksdb mapping the transaction mechanism from Guillotina. It needs to provide the same API from internal storage Guillotina so each object is stored in a secure hierarchical structure.
Storing the indexing information on each transaction on RocksDB and Tantivy to support keyword indexes and full-text index.
Providing an interface to search ok keyword indexes and full-text search with the security checks.
Providing an interface to load security and pickles using Guillotina internal storage API.
Optionally, providing a Raft protocol to sync multiple instances of the object server and distribute the load would be great.