Module dittolive_ditto::store::query_builder

source ·
Expand description

The original data API for Ditto uses a builder syntax to execute queries.

NOTE: If you’re newly adopting Ditto into your project, we recommend you use the newer Ditto Query Language (DQL) API for interacting with Ditto data.

The QueryBuilder API provides method calls that may be chained together in order to query or mutate data in the Ditto store, as well as to subscribe to changes in data on another device in the Ditto mesh.

Here’s the general flow for querying documents using the QueryBuilder API:

  • Use ditto.store().collection("...")? and provide a collection name to receive a Collection handle for querying documents in that collection.

  • Then, use one of the “find” methods such as .find_all(), .find_by_id(...), .find(...), or .find_with_args(...) to create a PendingCursorOperation which will yield documents from the collection. This cursor object may then be used to do any of the following:

    • Use .exec() to execute the query immediately, returning a list of all documents matching the query.

    • Use .subscribe() to create a Subscription which syncs documents matching the query from other Ditto peers. The subscription will remain active until it’s dropped.

    • Use .observe_local(...) to register a callback which will be called any time documents matching the query are changed in the local store. Only when combined with an active Subscription will this callback fire when documents are changed on a remote peer.

§Example: Query all documents in a collection

use dittolive_ditto::prelude::*;

let documents = ditto.store().collection("cars")?.find_all().exec()?;
let typed_documents: Vec<serde_json::Value> = documents
    .into_iter()
    .flat_map(|doc| doc.typed::<serde_json::Value>().ok())
    .collect();
println!("Documents in 'cars': {typed_documents:?}");

§Example: Subscribe to and observe changes from peers

Use .observe_local(...) to register a callback before using .subscribe() to create a Subscription to ensure that all documents synced from a remote peer are seen by the observer callback:

use dittolive_ditto::prelude::*;

let cars = ditto.store().collection("cars")?;
let query = cars.find_all();

// Create a `LiveQuery` by registering a local observer
// This observer is called whenever documents matching the query
// are changed in this peer's local store.
//
// Dropping this handle will cancel the observer
let _live_query = query.observe_local(|docs, _event| {
    let typed_docs: Vec<serde_json::Value> = docs
        .into_iter()
        .flat_map(|doc| doc.typed::<serde_json::Value>().ok())
        .collect();
    println!("Observed updated document: {typed_docs:?}");
})?;

// Create a `Subscription` which syncs documents from other peers
//
// Dropping this handle will cancel the subscription
let _subscription = query.subscribe();

§Ditto CRDT Types

The Ditto CRDT types are DittoCounter, DittoMutableCounter, DittoRegister, and DittoMutableRegister. These types have dedicated CRDT logic and specific behavior for merging concurrent updates. The mutable variants allow for updating the CRDT values, whereas the non-mutable variants are read-only.

To obtain a read-only CRDT, use the .get::<T>("...") API of DittoDocument. To obtain a mutable CRDT, use the .get_mut::<T>("...") API of DittoMutDocument. Both of these traits are implemented for any type that Derefs to Document, such as the BoxedDocuments returned by .exec().

§Counter

The DittoCounter and DittoMutableCounter represent counters which can be updated simultaneously on several peers at once without losing concurrent increments.

§Example

use dittolive_ditto::prelude::*;

let collection = ditto.store().collection("foo")?;

// Insert a new document with a field "counter" containing
// a `DittoCounter` with default value 0
let id = collection.upsert(serde_json::json!({
    "counter": DittoCounter::new()
}))?;

// Find the document by its ID and `.get` the "counter" field.
let doc = collection.find_by_id(&id).exec()?;
let counter: DittoCounter = doc.get("counter")?;
assert_eq!(counter.value(), 0.0);

// Update the document by ID and increment "counter" as a DittoMutableCounter
collection.find_by_id(&id).update(|maybe_doc| {
    let doc = maybe_doc.unwrap();
    let mut counter = doc.get_mut::<DittoMutableCounter>("counter").unwrap();
    counter.increment(5.0).unwrap();
})?;

let counter: DittoCounter = doc.get("counter")?;
assert_eq!(counter.value(), 5.0);

§Register

DittoRegister and DittoMutableRegister are containers for a single value following the concept of LWW: “Last Write Wins”. Primitive types such as String, u32, bool, etc., are implicitly wrapped to and unwrapped from registers.

§Example: Primitives and Registers

use dittolive_ditto::prelude::*;

let collection = ditto.store().collection("register_tests")?;

// Insert a new document with "register" and "string" fields with these values:
let id = collection.upsert(serde_json::json!({
    "register": DittoRegister::new(42)?,
    "string": "SomeString"
}))?;

// Find the document by its ID and `.get` the "register" field
let doc = collection.find_by_id(&id).exec()?;
let register: DittoRegister = doc.get("register")?;
assert_eq!(register.value::<u32>()?, 42);

// Use a register to access a primitive type such as String
let register: DittoRegister = doc.get("string")?;
assert_eq!(register.value::<String>()?, "SomeString");

// Update the "register" field by setting it directly to a value
collection
    .find_by_id(&id)
    .update(|maybe_doc| {
        let doc = maybe_doc.unwrap();
        let mut register = doc.get_mut::<DittoMutableRegister>("register").unwrap();
        register.set(5.0).unwrap();
    })?;

// Read back the "register" field and observe the changed value
let register: DittoRegister = doc.get("register")?;
assert_eq!(register.value::<f32>()?, 5.0);

// Read the primitive value directly via `.get`
let direct_access: f32 = doc.get("register")?;
assert_eq!(direct_access, 5.0);

§Example: Mixed fields without Registers

The main purpose of the DittoRegister type is to treat structs as a single CRDT value. Without Register, fields can be merged together and get mixed up. Consider this example where we would like to replace a Car vehicle and all of its fields with a Boat vehicle and its fields:

use dittolive_ditto::prelude::*;
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize)]
struct Car {
    color: String,
    price: u32,
}
#[derive(Serialize, Deserialize)]
struct Boat {
    length: f32,
    name: String,
}

let collection = ditto.store().collection("vehicles")?;

// Insert a car as a new document, returning the assigned ID
let car = Car {
    color: String::from("red"),
    price: 42_000,
};
let id = collection.upsert(serde_json::json!({ "vehicle": car }))?;

// Now, attempt to replace the Car with a Boat using `.update`
collection
    .find_by_id(&id)
    .update(|maybe_doc| {
        let doc = maybe_doc.unwrap();

        // Attempt to replace the value of the "vehicle" field with a Boat
        let boat = Boat {
            length: 248.0,
            name: String::from("Richelieu"),
        };
        doc.set("vehicle", boat).unwrap();
    })?;

// Uh oh, instead of replacing the vehicle, we just merged fields into it!
let document = collection.find_by_id(&id).exec()?;
let typed = document.typed::<serde_json::Value>()?;
println!("Vehicle document: {typed:#}");
// Vehicle document: {
//   "vehicle": {
//     "color": "red",
//     "length": 248,
//     "name": "Richelieu",
//     "price": 42_000
//   }
// }

To solve this problem where fields get mixed together, we can use a DittoRegister, which treats a block of fields as a single unbreakable unit. In this scenario, the original Car value will be lost, but the Boat value will successfully replace it.

use dittolive_ditto::prelude::*;
use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize)]
struct Car {
    color : String,
    price: u32,
}

#[derive(Serialize, Deserialize)]
struct Boat {
    length : f32,
    name: String,
}

let collection = ditto.store().collection("vehicles")?;

// Insert the Car wrapped in a `DittoRegister`
let car = Car { color: "red".to_string(), price: 42_000 };
let id = collection.upsert(serde_json::json!({
    "vehicle": DittoRegister::new(car)?
}))?;

// Update the "vehicle" field, replacing the Car with a Boat
collection
    .find_by_id(&id)
    .update(|maybe_doc| {
        let doc = maybe_doc.unwrap();

        let boat = Boat { length: 248.0, name: "Richelieu".to_string() };
        let mut register = doc.get_mut::<DittoMutableRegister>("vehicle").unwrap();
        register.set(boat).unwrap();
})?;

// Now, the fields in "vehicle" have been replaced by Boat rather than merged with Car!
let document = collection.find_by_id(&id).exec()?;
let typed = document.typed::<serde_json::Value>()?;
println!("Vehicle document: {typed:#}");
// Vechicle document: {
//   "vehicle": {
//     "length": 248,
//     "name" : "Richelieu"
//   }
// }

§Legacy Query Syntax

Certain QueryBuilder APIs accept query strings, and it’s important to note that these APIs DO NOT accept DQL, but rather a legacy query syntax. The following APIs use the legacy query syntax described in this section:

For a full overview of the Legacy Query Syntax, see the official Ditto docs Query Syntax page. We’ll briefly summarize the same info here.

§Boolean and Comparison operators

The objective of the query syntax is to select a subset of documents in a collection. Here are some examples of operators that may be used to filter documents based on fields they contain and the values of those fields:

  • Equality (==): Check that a field matches a specific value:

    • "isDeleted == true"
    • "isDeleted == false"
    • "title == 'Harry Potter'"

  • Inequality (!=): Check that a field does not match a specific value:

    • "title != 'Lord of the Rings'"

  • Less-than (<), Greater-than (>), Less-or-equal (<=), Greater-or-equal (>=):

    • "age < 18"
    • "age <= 18"
    • "age > 18"
    • "age >= 18"

§Compound Operators

Compound operators allow you to add multiple conditions into a single query.

  • Logical AND (&&): Evaluates to true when both child conditions are true:

    • "theme == 'Dark' && name == 'Light'"

  • Logical OR (||): Evaluates to true when at least one child condition is true:

    • "name == 'Tom' || name == 'Arthur'"

  • Logical NOT (!): Inverts the condition:

    • "!(name == 'Hamilton' || name == 'Morten')"

§String Operations

The legacy query syntax supports a handful of functions that may be used to do string comparisons against document field values.

  • Use starts_with(field, test) to test if the field begins with the test string

    • "starts_with(title, 'Lord')"

  • Use ends_with(field, test) to test if the field ends with the test string

    • "ends_with(title, 'Rings')"

  • Use regex(field, '<regex>') to test if a field matches the provided regular expression

    • "regex(title, '^([A-Za-z]|[0-9]|_)+$')"

§NULL values

Use the null primitive to check for the existence of a value at a given field

  • To find documents with a color property that has no value:
    • "color == null"

§Array Operations

When handling collections of data that different peers may make concurrent updates to, first consider using an embedded map in your documents. If this doesn’t work for your use-case, then try using an array.

The array type in Ditto is a CRDT and behaves differently than a typical array type. For more information, see the Platform Manual on Data Types.

We recommend avoiding arrays in Ditto where possible.

Due to Ditto’s use of CRDTs, arrays are much less graceful than maps when syncing with other peers and merging concurrent updates.

  • Use contains(array, value) to check if the value exists in the array
    • "contains(['blue', 'green'], color)"

§Date and Time formats

When querying or parsing date and time strings, use the ISO-8601 standard format.

  • "created_at >= '2022-04-29T00:55:31.859Z'"

§Field Path Navigation

If a field name consists of alphanumeric characters or includes underscores, use any of the following notations to navigate document properties:

  • For fields that are alphanumeric or include underscores, use dot notation:

    • "name.last == 'Turing'"

  • For all other scenarios, use bracket notation:

    • "work['street-line'] == '678 Johnson Street'"

Structs§

Enums§

Traits§