Get excited

A reactive database framework

Build powerful React and React Native apps that scale from hundreds to tens of thousands of records and remain fast ⚡️

Why Watermelon?

WatermelonDB is a new way of dealing with user data in React Native and React web apps.

It's optimized for building complex applications in React Native, and the number one goal is real-world performance. In simple words, your app must launch fast.

For simple apps, using Redux or MobX with a persistence adapter is the easiest way to go. But when you start scaling to thousands or tens of thousands of database records, your app will now be slow to launch (especially on slower Android devices). Loading a full database into JavaScript is expensive!

Watermelon fixes it by being lazy. Nothing is loaded unless requested. And since all querying is performed directly on the rock-solid SQLite database on a separate native thread, most queries resolve in an instant.

But unlike using SQLite directly, Watermelon is fully observable. So whenever you change a record, all UI that depends on it will automatically re-render. For example, completing a task in a to-do app will re-render the task component, the list (to reorder), and all relevant task counters. Learn more.

📺 Next-generation React databases (a talk about WatermelonDB)	✨ Check out the Demo

Usage

Quick (over-simplified) example: an app with posts and comments.

First, you define Models:

class Post extends Model {
  @field('name') name
  @field('body') body
  @children('comments') comments
}

class Comment extends Model {
  @field('body') body
  @field('author') author
}

Then, you connect components to the data:

const Comment = ({ comment }) => (
  <View style={styles.commentBox}>
    <Text>{comment.body} — by {comment.author}</Text>
  </View>
)

// This is how you make your app reactive! ✨
const enhance = withObservables(['comment'], ({ comment }) => ({
  comment: comment.observe()
}))
const EnhancedComment = enhance(Comment)

And now you can render the whole Post:

const Post = ({ post, comments }) => (
  <View>
    <Text>{post.name}</Text>
    <Text>Comments:</Text>
    {comments.map(comment =>
      <Comment key={comment.id} comment={comment} />
    )}
  </View>
)

const enhance = withObservables(['post'], ({ post }) => ({
  post,
  comments: post.comments
}))

The result is fully reactive! Whenever a post or comment is added, changed, or removed, the right components will automatically re-render on screen. Doesn't matter if a change occurred in a totally different part of the app, it all just works out of the box!

➡️ Learn more: see full documentation

Who uses WatermelonDB

Does your company or app use 🍉? Open a pull request and add your logo/icon with link here!

Contributing

WatermelonDB is an open-source project and it needs your help to thrive!

If there's a missing feature, a bug, or other improvement you'd like, we encourage you to contribute! Feel free to open an issue to get some guidance and see Contributing guide for details about project setup, testing, etc.

If you're just getting started, see good first issues that are easy to contribute to. If you make a non-trivial contribution, email me, and I'll send you a nice 🍉 sticker!

If you make or are considering making an app using WatermelonDB, please let us know!

Author and license

WatermelonDB was created by @Nozbe. Main author and maintainer is Radek Pietruszewski.

Contributors: @mobily, @kokusGr, @rozPierog, @rkrajewski, @domeknn, @Tereszkiewicz and more.

WatermelonDB is available under the MIT license. See the LICENSE file for more info.

Demo

See how WatermelonDB performs at large scales in the demo app.

Online demo

Check out WatermelonDB demo online

Note that where Watermelon really shines is in React Native apps — see instructions below ⬇️

Running React Native demo

To compile the WatermelonDB demo on your own machine:

1. Install React Native toolkit if you haven't already
2. Download this project

   git clone https://github.com/Nozbe/WatermelonDB.git
   cd WatermelonDB/examples/native
   yarn
   

3. Run the React Native packager:

   yarn dev
   

4. Run the app on iOS or Android:

   yarn start:ios # or:
   yarn start:android
   

⚠️ Note that for accurate measurement of performance, you need to compile the demo app in Release mode and run it on a real device, not the simulator.

⚠️ If iOS app doesn't compile, try running it from Xcode instead of the terminal first

⚠️ You might want to git checkout the latest stable tag if the demo app doesn't work

Running web demo

To compile the WatermelonDB demo on your own machine:

1. Download this project

   git clone https://github.com/Nozbe/WatermelonDB.git
   cd WatermelonDB/examples/web
   yarn
   

2. Run the server:

   yarn dev
   

3. Webpack will point you to the right URL to open in the browser

You can also use Now to deploy the demo app (requires a Zeit account):

now

⚠️ You might want to git checkout the latest stable tag if the demo app doesn't work

Learn to use Watermelon

Learn the basics of how to use WatermelonDB

Installation

First, add Watermelon to your project:

yarn add @nozbe/watermelondb
yarn add @nozbe/with-observables

or alternatively if you prefer npm:

npm install @nozbe/watermelondb
npm install @nozbe/with-observables

React Native setup

1. Install the Babel plugin for decorators if you haven't already:

   yarn add --dev @babel/plugin-proposal-decorators
   

   or

   npm install -D @babel/plugin-proposal-decorators
   
   

2. Add ES6 decorators support to your .babelrc file:

   {
     "presets": ["module:metro-react-native-babel-preset"],
     "plugins": [
       ["@babel/plugin-proposal-decorators", { "legacy": true }]
     ]
   }
   

3. Set up your iOS or Android project — see instructions below

iOS (React Native)

1. Set up Babel config in your project

   See instructions above ⬆️

2. Add Swift support to your Xcode project:

   • Open ios/YourAppName.xcodeproj in Xcode
   • Right-click on Your App Name in the Project Navigator on the left, and click New File…
   • Create a single empty Swift file to the project (make sure that Your App Name target is selected when adding), and when Xcode asks, press Create Bridging Header and do not remove Swift file then.

3. Link WatermelonDB's native library with the Xcode project:

   Automatically

   react-native link @nozbe/watermelondb
   

   Or manually

   If you don't want to use react-native link, you can link the library manually:

   1. Open your project in Xcode, right click on Libraries in the Project Navigator on the left and click Add Files to "Your Project Name". Look under node_modules/@nozbe/watermelondb/native/ios and select WatermelonDB.xcodeproj
   2. Go to Project settings (top item in the Project navigator on the left), select your app name under Targets → Build Phases → Link Binary With Libraries, and add libWatermelonDB.a

   For more information about linking libraries manually, see React Native documentation.

   Using CocoaPods

   Please contribute!

Note that Xcode 9.4 and a deployment target of at least iOS 9.0 is required (although Xcode 10 and iOS 11.0 are recommended).

Android (React Native)

1. Set up Babel config in your project

   See instructions above ⬆️

2. In android/settings.gradle, add:

   include ':watermelondb'
   project(':watermelondb').projectDir =
       new File(rootProject.projectDir, '../node_modules/@nozbe/watermelondb/native/android')
   

3. In android/app/build.gradle, add:

   apply plugin: "com.android.application"
   apply plugin: 'kotlin-android'  // ⬅️ This!
   // ...
   dependencies {
       // ...
       implementation project(':watermelondb')  // ⬅️ This!
   }
   

4. In android/build.gradle, add Kotlin support to the project:

   buildscript {
       ext.kotlin_version = '1.3.21'
       // ...
       dependencies {
           // ...
           classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
       }
   }
   

5. And finally, in android/app/src/main/java/{YOUR_APP_PACKAGE}/MainApplication.java, add:

   // ...
   import com.nozbe.watermelondb.WatermelonDBPackage; // ⬅️ This!
   // ...
   @Override
   protected List<ReactPackage> getPackages() {
     return Arrays.<ReactPackage>asList(
       new MainReactPackage(),
       new WatermelonDBPackage() // ⬅️ Here!
     );
   }
   

6. Troubleshooting. If you get this error:

   Can't find variable: Symbol

   You might need a polyfill for ES6 Symbol:

   yarn add es6-symbol
   

   And in your index.js:

   import 'es6-symbol/implement'
   

   Alternatively, we also recommend jsc-android, with which you don't need this polyfill, and it also makes your app faster.

Web setup

This guide assumes you use Webpack as your bundler.

3. If you haven't already, install Babel plugins for decorators, static class properties, and async/await to get the most out of Watermelon. This assumes you use Babel 7 and already support ES6 syntax.

   yarn add --dev @babel/plugin-proposal-decorators
   yarn add --dev @babel/plugin-proposal-class-properties
   yarn add --dev @babel/plugin-transform-runtime
   

   or

   npm install -D @babel/plugin-proposal-decorators
   npm install -D @babel/plugin-proposal-class-properties
   npm install -D @babel/plugin-transform-runtime
   

4. Add ES7 support to your .babelrc file:

   {
     "plugins": [
       ["@babel/plugin-proposal-decorators", { "legacy": true }],
       ["@babel/plugin-proposal-class-properties", { "loose": true }],
       [
         "@babel/plugin-transform-runtime",
          {
            "helpers": true,
            "regenerator": true
          }
       ]
     ]
   }
   

If you want to use Web Worker for WatermelonDB (this has pros and cons, we recommend you start without Web Workers, and evaluate later if it makes sense for your app to use them):

1. Install worker-loader Webpack plugin to add support for Web Workers to your app:

   yarn add --dev worker-loader
   

   or

   npm install -D worker-loader
   

2. And add this to Webpack configuration:

   // webpack.config.js
   {
     module: {
       rules: [
         // ⬇️ Add this:
         {
           test: /\.worker\.js$/,
           use: { loader: 'worker-loader' }
         }
       ]
     },
     // ...
     output: {
       // ...
       globalObject: 'this', // ⬅️ And this
     }
   }
   

Set up Database

Create model/schema.js in your project:

import { appSchema, tableSchema } from '@nozbe/watermelondb'

export default appSchema({
  version: 1,
  tables: [
    // tableSchemas go here...
  ]
})

You'll need it for the next step. Now, in your index.js:

import { Database } from '@nozbe/watermelondb'
import SQLiteAdapter from '@nozbe/watermelondb/adapters/sqlite'

import schema from './model/schema'
// import Post from './model/Post' // ⬅️ You'll import your Models here

// First, create the adapter to the underlying database:
const adapter = new SQLiteAdapter({
  schema,
})

// Then, make a Watermelon database from it!
const database = new Database({
  adapter,
  modelClasses: [
    // Post, // ⬅️ You'll add Models to Watermelon here
  ],
  actionsEnabled: true,
})

The above will work on iOS and Android (React Native). For the web, instead of SQLiteAdapter use LokiJSAdapter:

import LokiJSAdapter from '@nozbe/watermelondb/adapters/lokijs'

const adapter = new LokiJSAdapter({
  schema,
  // These two options are recommended for new projects:
  useWebWorker: false,
  useIncrementalIndexedDB: true,
  // It's recommended you implement this method:
  // onIndexedDBVersionChange: () => {
  //   // database was deleted in another browser tab (user logged out), so we must make sure we delete
  //   // it in this tab as well
  //   if (checkIfUserIsLoggedIn()) {
  //     window.location.reload()
  //   }
  // },
})

// The rest is the same!

Next steps

➡️ After Watermelon is installed, define your app's schema

Schema

When using WatermelonDB, you're dealing with Models and Collections. However, underneath Watermelon sits an underlying database (SQLite or LokiJS) which speaks a different language: tables and columns. Together, those are called a database schema and we must define it first.

Defining a Schema

Say you want Models Post, Comment in your app. For each of those Models, you define a table. And for every field of a Model (e.g. name of the blog post, author of the comment) you define a column. For example:

// model/schema.js
import { appSchema, tableSchema } from '@nozbe/watermelondb'

export const mySchema = appSchema({
  version: 1,
  tables: [
    tableSchema({
      name: 'posts',
      columns: [
        { name: 'title', type: 'string' },
        { name: 'subtitle', type: 'string', isOptional: true },
        { name: 'body', type: 'string' },
        { name: 'is_pinned', type: 'boolean' },
      ]
    }),
    tableSchema({
      name: 'comments',
      columns: [
        { name: 'body', type: 'string' },
        { name: 'post_id', type: 'string', isIndexed: true },
      ]
    }),
  ]
})

Note: It is database convention to use plural and snake_case names for table names. Column names are also snake_case. So Post become posts and createdAt becomes created_at.

Column types

Columns have one of three types: string, number, or boolean.

Fields of those types will default to '', 0, or false respectively, if you create a record with a missing field.

To allow fields to be null, mark the column as isOptional: true.

Naming conventions

To add a relation to a table (e.g. Post where a Comment was published, or author of a comment), add a string column ending with _id:

{ name: 'post_id', type: 'string' },
{ name: 'author_id', type: 'string' },

Boolean columns should have names starting with is_:

{ name: 'is_pinned', type: 'boolean' }

Date fields should be number (dates are stored as Unix timestamps) and have names ending with _at:

{ name: 'last_seen_at', type: 'number', isOptional: true }

Special columns

All tables automatically have a string column id to uniquely identify records. (Also two special columns for sync purposes). You can add special created_at / updated_at columns to enable automatic create/update tracking.

Modifying Schema

Whenever you change the Schema, you must increment the version number. During development, this will cause the database to clear completely on next launch.

To seamlessly change the schema (without deleting data), use Migrations.

⚠️ Always use Migrations if you already shipped your app.

Indexing

To enable database indexing, add isIndexed: true to a column.

Indexing makes querying by a column faster, at the slight expense of create/update speed and database size.

For example, you will often want to query all comments belonging to a post (that is, query comments by its post_id column), and so you should mark the post_id column as indexed.

However, if you rarely query all comments by its author, indexing author_id is probably not worth it.

In general, most _id fields are indexed. Sometimes, boolean fields are worth indexing if you often use it for queries. However, you should almost never index date (_at) columns or string columns.

Next steps

➡️ After you define your schema, go ahead and define your Models

Defining Models

A Model class represents a type of thing in your app. For example, Post, Comment, User.

Before defining a Model, you first need to define its schema.

Create a Model

Let's define the Post model:

// model/Post.js
import { Model } from '@nozbe/watermelondb'

export default class Post extends Model {
  static table = 'posts'
}

Mark the table name for this Model — the same you defined in the schema.

Now add the new Model to Database:

// index.js
import Post from 'model/Post'

const database = new Database({
  // ...
  modelClasses: [Post],
})

Associations

Your models almost surely relate to one another. A Post has many Comments. And every Comment belongs to a Post. (Every relation is double-sided). Define those associations like so:

class Post extends Model {
  static table = 'posts'
  static associations = {
    comments: { type: 'has_many', foreignKey: 'post_id' },
  }
}

class Comment extends Model {
  static table = 'comments'
  static associations = {
    posts: { type: 'belongs_to', key: 'post_id' },
  }
}

On the "child" side (comments) you define a belongs_to association, and pass a column name (key) that points to the parent (post_id is the ID of the post the comment belongs to).

On the "parent" side (posts) you define an equivalent has_many association and pass the same column name (here named foreignKey).

Add fields

Next, define the Model's fields (properties). Those correspond to table columns defined earlier in the schema.

import { field } from '@nozbe/watermelondb/decorators'

class Post extends Model {
  static table = 'posts'
  static associations = {
    comments: { type: 'has_many', foreignKey: 'post_id' },
  }

  @field('title') title
  @field('body') body
  @field('is_pinned') isPinned
}

Fields are defined using ES6 decorators. Pass column name you defined in Schema as the argument to @field.

Field types. Fields are guaranteed to be the same type (string/number/boolean) as the column type defined in Schema. If column is marked isOptional: true, fields may also be null.

Note: Why do I have to type the field/column name twice? The database convention is to use snake_case for names, and the JavaScript convention is to use camelCase. So for any multi-word name, the two differ. Also, for resiliency, we believe it's better to be explicit, because over time, you might want to refactor how you name your JavaScript field names, but column names must stay the same for backward compatibility.

Date fields

For date fields, use @date instead of @field. This will return a JavaScript Date object (instead of Unix timestamp integer).

import { date } from '@nozbe/watermelondb/decorators'

class Post extends Model {
  // ...
  @date('last_event_at') lastEventAt
}

Relation fields

To-one relation

To point to a related record, e.g. Post a Comment belongs to, or author (User) of a Comment, use @relation:

import { relation } from '@nozbe/watermelondb/decorators'

class Comment extends Model {
  // ...
  @relation('posts', 'post_id') post
  @relation('users', 'author_id') author
}

➡️ Learn more: Relation API

Children (To-Many relation)

To point to a list of records that belong to this Model, e.g. all Comments that belong to a Post, you can define a simple Query using @children:

import { children } from '@nozbe/watermelondb/decorators'

class Post extends Model {
  static table = 'posts'
  static associations = {
    comments: { type: 'has_many', foreignKey: 'post_id' },
  }

  @children('comments') comments
}

Pass the table name of the related records as an argument to @children. The resulting property will be a Query you can fetch, observe, or count.

Note: You must define a has_many association in static associations for this to work

➡️ Learn more: Queries

Advanced

Actions

Define actions to simplify creating and updating records.

➡️ Learn more: Actions

Queries

In addition to @children, you can define custom Queries or extend existing ones.

➡️ Learn more: Queries

Advanced fields

You can also use these decorators:

• @text trims whitespace from user-input text
• @json for complex serialized data
• @readonly to make the field read-only
• @nochange to disallow changes to the field after the first creation

➡️ Learn more: Advanced fields

Next steps

➡️ After you define some Models, learn the Create / Read / Update / Delete API

Create, Read, Update, Delete

When you have your Schema and Models defined, learn how to manipulate them!

Collections

The Collection object is how you find, query, and create new records of a given type.

Get a collection

const postsCollection = database.collections.get('posts')

// Shortcut syntax:
const postsCollection = database.get('posts')

Pass the table name as the argument.

Find a record (by ID)

const post = await postsCollection.find('abcdef')

find() returns a Promise. If the record cannot be found, the Promise will be rejected.

Query records

Find a list of records matching given conditions using .query():

const allPosts = await postsCollection.query().fetch()
const starredPosts = await postsCollection.query(Q.where('is_starred', true)).fetch()

➡️ Learn more: Queries

Modifying the database

To create, update, or delete records, use the respective operations wrapped in an Action:

await database.action(async () => {
  const post = await postsCollection.find('abcdef')
  await post.update( /* update the post */ )
  await post.markAsDeleted()
})

➡️ Learn more: Actions

Create a new record

await database.action(async () => {
  const newPost = await postsCollection.create(post => {
    post.title = 'New post'
    post.body = 'Lorem ipsum...'
  })
})

.create() takes a "builder function". In the example above, the builder will get a Post object as an argument. Use this object to set values for fields you defined.

Note: Always await the Promise returned by create before you access the created record.

Note: You can only use field setters in create() or update() builder functions.

Update a record

await database.action(async () => {
  await somePost.update(post => {
    post.title = 'Updated title'
  })
})

Like creating, updating takes a builder function, where you can use field setters.

Note: Always await the Promise returned by update before you access the modified record.

Delete a record

There are two ways of deleting records: syncable (mark as deleted), and permanent.

If you only use Watermelon as a local database, destroy records permanently, if you synchronize, mark as deleted instead.

await database.action(async () => {
  await somePost.markAsDeleted() // syncable
  await somePost.destroyPermanently() // permanent
})

Note: Don't access, update, or observe records after they're destroyed.

Advanced

• Model.observe() - usually you only use this when connecting records to components, but you can manually observe a record outside of React components. The returned RxJS Observable will emit the record immediately upon subscription, and then every time the record is updated. If the record is deleted, the Observable will complete.
• Query.observe(), Relation.observe() — analagous to the above, but for Queries and Relations
• Query.observeWithColumns() - used for sorted lists
• Collection.findAndObserve(id) — same as using .find(id) and then calling record.observe()
• Model.prepareUpdate(), Collection.prepareCreate, Database.batch — used for batch updates
• Database.unsafeResetDatabase() destroys the whole database - be sure to see this comment before using it
• To override the record.id during the creation, e.g. to sync with a remote database, you can do it by record._raw property. Be aware that the id must be of type string.

await postsCollection.create(post => {
  post._raw.id = serverId
})

Next steps

➡️ Now that you can create and update records, connect them to React components

Connecting to Components

After you define some Models, it's time to connect Watermelon to your app's interface. We're using React in this guide.

Install withObservables

The recommended way to use Watermelon with React is with withObservables HOC (higher-order component). It doesn't come pre-packaged with Watermelon, but you can install it with:

yarn add @nozbe/with-observables

Note: If you're not familiar with higher-order components, read React documentation, check out recompose… or just read the examples below to see it in practice!

Reactive components

Here's a very simple React component rendering a Comment record:

const Comment = ({ comment }) => (
  <div>
    <p>{comment.body}</p>
  </div>
)

Now we can fetch a comment: const comment = await commentsCollection.find(id) and then render it: <Comment comment={comment} />. The only problem is that this is not reactive. If the Comment is updated or deleted, the component will not re-render to reflect the changes. (Unless an update is forced manually or the parent component re-renders).

Let's enhance the component to make it observe the Comment automatically:

const enhance = withObservables(['comment'], ({ comment }) => ({
  comment // shortcut syntax for `comment: comment.observe()`
}))
const EnhancedComment = enhance(Comment)

Now, if we render <EnhancedComment comment={comment} />, it will update every time the comment changes.

Reactive lists

Let's render the whole Post with comments:

import withObservables from '@nozbe/with-observables'

const Post = ({ post, comments }) => (
  <article>
    <h1>{post.name}</h1>
    <p>{post.body}</p>
    <h2>Comments</h2>
    {comments.map(comment =>
      <EnhancedComment key={comment.id} comment={comment} />
    )}
  </article>
)

const enhance = withObservables(['post'], ({ post }) => ({
  post,
  comments: post.comments, // Shortcut syntax for `post.comments.observe()`
}))

const EnhancedPost = enhance(Post)

Notice a couple of things:

1. We're starting with a simple non-reactive Post component

2. Like before, we enhance it by observing the Post. If the post name or body changes, it will re-render.

3. To access comments, we fetch them from the database and observe using post.comments.observe() and inject a new prop comments. (post.comments is a Query created using @children).

   Note that we can skip .observe() and just pass post.comments for convenience — withObservables will call observe for us

4. By observing the Query, the <Post> component will re-render if a comment is created or deleted

5. However, observing the comments Query will not re-render <Post> if a comment is updated — we render the <EnhancedComment> so that it observes the comment and re-renders if necessary.

Reactive relations

The <Comment> component we made previously only renders the body of the comment but doesn't say who posted it.

Assume the Comment model has a @relation('users', 'author_id') author field. Let's render it:

const Comment = ({ comment, author }) => (
  <div>
    <p>{comment.body} — by {author.name}</p>
  </div>
)

const enhance = withObservables(['comment'], ({ comment }) => ({
  comment,
  author: comment.author, // shortcut syntax for `comment.author.observe()`
}))
const EnhancedComment = enhance(Comment)

comment.author is a Relation object, and we can call .observe() on it to fetch the User and then observe changes to it. If author's name changes, the component will re-render.

Note again that we can also pass Relation objects directly for convenience, skipping .observe()

Reactive counters

Let's make a <PostExcerpt> component to display on a list of Posts, with only a brief summary of the contents and only the number of comments it has:

const PostExcerpt = ({ post, commentCount }) => (
  <div>
    <h1>{post.name}</h1>
    <p>{getExcerpt(post.body)}</p>
    <span>{commentCount} comments</span>
  </div>
)

const enhance = withObservables(['post'], ({ post }) => ({
  post: post.observe(),
  commentCount: post.comments.observeCount()
}))

const EnhancedPostExcerpt = enhance(PostExcerpt)

This is very similar to normal <Post>. We take the Query for post's comments, but instead of observing the list of comments, we call observeCount(). This is far more efficient. And as always, if a new comment is posted, or one is deleted, the component will re-render with the updated count.

Hey, what about React Hooks?

We get it — HOCs are so 2017, and Hooks are the future! And we agree.

Instead of using withObservables HOC you can use an alternative open-source Hook for Rx Observables. But be warned that they are probably not as optimized for performance and WatermelonDB use as withObservables.

If you'd like to see official useObservables Hook - please contribute ❤️

Understanding withObservables

Let's unpack this:

withObservables(['post'], ({ post }) => ({
  post: post.observe(),
  commentCount: post.comments.observeCount()
}))

1. Starting from the second argument, ({ post }) are the input props for the component. Here, we receive post prop with a Post object.
2. These:

   ({
     post: post.observe(),
     commentCount: post.comments.observeCount()
   })
   

   are the enhanced props we inject. The keys are props' names, and values are Observable objects. Here, we override the post prop with an observable version, and create a new commentCount prop.
3. The first argument: ['post'] is a list of props that trigger observation restart. So if a different post is passed, that new post will be observed. If you pass [], the rendered Post will not change. You can pass multiple prop names if any of them should cause observation to re-start.
4. Rule of thumb: If you want to use a prop in the second arg function, pass its name in the first arg array

Advanced

1. findAndObserve. If you have, say, a post ID from your Router (URL in the browser), you can use:

   withObservables(['postId'], ({ postId, database }) => ({
     post: database.collections.get('posts').findAndObserve(postId)
   }))
   

2. RxJS transformations. The values returned by Model.observe(), Query.observe(), Relation.observe() are RxJS Observables. You can use standard transforms like mapping, filtering, throttling, startWith to change when and how the component is re-rendered.
3. Custom Observables. withObservables is a general-purpose HOC for Observables, not just Watermelon. You can create new props from any Observable.

Advanced: observing sorted lists

If you have a list that's dynamically sorted (e.g. sort comments by number of likes), use Query.observeWithColumns to ensure the list is re-rendered when its order changes:

// This is a function that sorts an array of comments according to its `likes` field
// I'm using `ramda` functions for this example, but you can do sorting however you like
const sortComments = sortWith([
  descend(prop('likes'))
])

const CommentList = ({ comments }) => (
  <div>
    {sortComments(comments).map(comment =>
      <EnhancedComment key={comment.id} comment={comment} />
    )}
  </div>
)

const enhance = withObservables(['post'], ({ post }) => ({
  comments: post.comments.observeWithColumns(['likes'])
}))

const EnhancedCommentList = enhance(CommentList)

If you inject post.comments.observe() into the component, the list will not re-render to change its order, only if comments are added or removed. Instead, use query.observeWithColumns() with an array of column names you use for sorting to re-render whenever a record on the list has any of those fields changed.

Advanced: observing 2nd level relations

If you have 2nd level relations, like author's Contact info, and want to connect it to a component as well, you cannot simply use post.author.contact.observe() in withComponents. Before accessing and observing the Contact relation, you need to resolve the author itself. Here is the simplest way to do it:

const enhancePostAndAuthor = withObservables(['post'], ({post}) => ({
  post,
  author: post.author,
}));

const enhanceAuthorContact = withObservables(['author'], ({author}) => ({
  contact: author.contact,
}));

const EnhancedPost = enhancePostAndAuthor(enhanceAuthorContact(PostComponent));

If you are familiar with rxjs, another way to achieve the same result is using switchMap operator:

import { switchMap } from 'rxjs/operators'

const enhancePost = withObservables(['post'], ({post}) => ({
  post: post,
  author: post.author,
  contact: post.author.observe().pipe(switchMap(author => author.contact.observe()))
}));

const EnhancedPost = enhancePost(PostComponent);

Now PostComponent will have Post, Author and Contact props.

Note: If you have an optional relation between Post and Author, the enhanceAuthorContact might receive null as author prop. For this case, as you must always return an observable for the contact prop, you can use rxjs of function to create a default or empty Contact prop:

import { of as of$ } from 'rxjs';


const enhanceAuthorContact = withObservables(['author'], ({author}) => ({
  contact: author ? author.contact.observe() : of$(null)
}));

With the switchMap approach, you can obtain the same result by doing:

contact: post.autor.observe().pipe(switchMap(author => author ? autor.contact : of$(null)))

Database Provider

To prevent prop drilling you can utilise the Database Provider and the withDatabase Higher-Order Component.

import DatabaseProvider from '@nozbe/watermelondb/DatabaseProvider'

// ...

const database = new Database({
  adapter,
  modelClasses: [Blog, Post, Comment],
  actionsEnabled: true,
})

render(
  <DatabaseProvider database={database}>
    <Root />
  </DatabaseProvider>, document.getElementById('application')
)

To consume the database in your components you just wrap your component like so:

import { withDatabase } from '@nozbe/watermelondb/DatabaseProvider'

// ...

export default withDatabase(withObservables([], ({ database }) => ({
  blogs: database.collections.get('blogs').query().observe(),
}))(BlogList))

The database prop in the withObservables Higher-Order Component is provided by the database provider.

useDatabase

You can also consume Database object using React Hooks syntax:

import { useDatabase } from '@nozbe/watermelondb/hooks'

const Component = () => {
   const database = useDatabase()
}

Next steps

➡️ Next, learn more about custom Queries

Query API

Querying is how you find records that match certain conditions, for example:

• Find all comments that belong to a certain post
• Find all verified comments made by John
• Count all verified comments made by John or Lucy published under posts made in the last two weeks

Because queries are executed on the database, and not in JavaScript, they're really fast. It's also how Watermelon can be fast even at large scales, because even with tens of thousands of records total, you rarely need to load more than a few dozen records at app launch.

Defining Queries

@children

The simplest query is made using @children. This defines a Query for all comments that belong to a Post:

class Post extends Model {
  // ...
  @children('comments') comments
}

➡️ Learn more: Defining Models

Extended Query

To narrow down a Query (add extra conditions to an existing Query), use .extend():

import { children, lazy } from '@nozbe/watermelondb/decorators'

class Post extends Model {
  // ...
  @children('comments') comments
  @lazy verifiedComments = this.comments.extend(Q.where('is_verified', true))
  @lazy verifiedAwesomeComments = this.verifiedComments.extend(Q.where('is_awesome', true))
}

Note: Use the @lazy when extending or defining new Queries for performance

Custom Queries

You can query any table using this.collections.get(tableName).query(conditions). Here, post.comments will query all users that made a comment under post.

class Post extends Model {
  // ...
  @lazy commenters = this.collections.get('users').query(
    Q.on('comments', 'post_id', this.id)
  )
}

Executing Queries

Most of the time, you connect Queries to Components by using observe or observeCount:

withObservables(['post'], ({ post }) => ({
  post: post.observe(),
  comments: post.comments.observe(),
  verifiedCommentCount: post.verifiedComments.observeCount(),
}))

Fetch

To simply get the current list or current count, use fetch / fetchCount. You might need it in Actions.

const comments = await post.comments.fetch()
const verifiedCommentCount = await post.verifiedComments.fetchCount()

// Shortcut syntax:
const comments = await post.comments
const verifiedCommentCount = await post.verifiedComments.count

Query conditions

import { Q } from '@nozbe/watermelondb'
// ...
commentCollection.query(
  Q.where('is_verified', true)
)

This will query all comments that are verified (all comments with one condition: the is_verified column of a comment must be true).

When making conditions, you refer to column names of a table (i.e. is_verified, not isVerified). This is because queries are executed directly on the underlying database.

The second argument is the value we want to query for. Note that the passed argument must be the same type as the column (string, number, or boolean; null is allowed only if the column is marked as isOptional: true in the schema).

Empty query

const allComments = await commentCollection.query().fetch()

A Query with no conditions will find all records in the collection.

Note: Don't do this unless necessary. It's generally more efficient to only query the exact records you need.

Multiple conditions

commentCollection.query(
  Q.where('is_verified', true),
  Q.where('is_awesome', true)
)

This queries all comments that are both verified and awesome.

Conditions with other operators

Note: It's NOT SAFE to use Q.like and Q.notLike with user input directly, because special characters like % or _ are not escaped. Always sanitize user input like so:

Q.like(`%${Q.sanitizeLikeString(userInput)}%`)
Q.notLike(`%${Q.sanitizeLikeString(userInput)}%`)

You can use Q.like for search-related tasks. For example, to find all users whose username start with "jas" (case-insensitive) you can write

usersCollection.query(
  Q.where("username", Q.like(`${Q.sanitizeLikeString("jas")}%`)
)

where "jas" can be changed dynamically with user input.

Conditions on related tables

For example: query all comments under posts published by John:

commentCollection.query(
  Q.on('posts', 'author_id', john.id),
)

Normally you set conditions on the table you're querying. Here we're querying comments, but we have a condition on the post the comment belongs to.

The first argument for Q.on is the table name you're making a condition on. The other two arguments are same as for Q.where.

Note: The two tables must be associated before you can use Q.on.

Advanced Queries

Advanced observing

Call query.observeWithColumns(['foo', 'bar']) to create an Observable that emits a value not only when the list of matching records changes (new records/deleted records), but also when any of the matched records changes its foo or bar column. Use this for observing sorted lists

Count throttling

By default, calling query.observeCount() returns an Observable that is throttled to emit at most once every 250ms. You can disable throttling using query.observeCount(false).

AND/OR nesting

You can nest multiple conditions using Q.and and Q.or:

commentCollection.query(
  Q.where('archived_at', Q.notEq(null)),
  Q.or(
    Q.where('is_verified', true),
    Q.and(
      Q.where('likes', Q.gt(10)),
      Q.where('dislikes', Q.lt(5))
    )
  )
)

This is equivalent to archivedAt !== null && (isVerified || (likes > 10 && dislikes < 5)).

Column comparisons

This queries comments that have more likes than dislikes. Note that we're comparing likes column to another column instead of a value.

commentCollection.query(
  Q.where('likes', Q.gt(Q.column('dislikes')))
)

sortBy, take, skip

When using SQLite adapter, you can use these experimental clauses to sort the result of the query and to limit the number of results

commentCollection.query(
  Q.experimentalSortBy('likes', Q.asc), // sorts ascending by `likes`
  Q.experimentalSkip(100),
  Q.experimentalTake(100),
)

NOTE: This does not currently work on web/LokiJS (please contribute!), and causes query observation to fall back to a less efficient method. We recommend using sortBy only when you absolutely need to limit queries, otherwise, it may be better to sort in JavaScript.

Security

Remember that Queries are a sensitive subject, security-wise. Never trust user input and pass it directly into queries. In particular:

• Never pass into queries values you don't know for sure are the right type (e.g. value passed to Q.eq() should be a string, number, boolean, or null -- but not an Object. If the value comes from JSON, you must validate it before passing it!)
• Never pass column names (without whitelisting) from user input
• Values passed to oneOf, notIn should be arrays of simple types - be careful they don't contain objects
• Do not use Q.like / Q.notLike without Q.sanitizeLikeString
• Do not use unsafe raw queries without knowing what you're doing and sanitizing all user input

Raw Queries

If this Query syntax is not enough for you, and you need to get your hands dirty on a raw SQL or Loki query, you need rawQueries. For now, only record SQL queries are available. If you need other SQL queries or LokiJS raw queries, please contribute!

const records = commentCollection.unsafeFetchRecordsWithSQL('select * from comments where ...')

Please don't use this if you don't know what you're doing. The method name is called unsafe for a reason. You need to be sure to properly sanitize user values to avoid SQL injection, and filter out deleted records using where _status is not 'deleted' clause

null behavior

There are some gotchas you should be aware of. The Q.gt, gte, lt, lte, oneOf, notIn, like operators match the semantics of SQLite in terms of how they treat null. Those are different from JavaScript.

Rule of thumb: No null comparisons are allowed.

For example, if you query comments for Q.where('likes', Q.lt(10)), a comment with 8 likes and 0 likes will be included, but a comment with null likes will not! In Watermelon queries, null is not less than any number. That's why you should avoid making table columns optional unless you actually need it.

Similarly, if you query with a column comparison, like Q.where('likes', Q.gt(Q.column('dislikes'))), only comments where both likes and dislikes are not null will be compared. A comment with 5 likes and null dislikes will NOT be included. 5 is not greater than null here.

Q.oneOf operator: It is not allowed to pass null as an argument to Q.oneOf. Instead of Q.oneOf([null, 'published', 'draft']) you need to explicitly allow null as a value like so:

postsCollection.query(
  Q.or(
    Q.where('status', Q.oneOf(['published', 'draft'])),
    Q.where('status', null)
  )
)

Q.notIn operator: If you query, say, posts with Q.where('status', Q.notIn(['published', 'draft'])), it will match posts with a status different than published or draft, however, it will NOT match posts with status == null. If you want to include such posts, query for that explicitly like with the example above.

Q.weakGt operator: This is weakly typed version of Q.gt — one that allows null comparisons. So if you query comments with Q.where('likes', Q.weakGt(Q.column('dislikes'))), it WILL match comments with 5 likes and null dislikes. (For weakGt, unlike standard operators, any number is greater than null).

Next steps

➡️ Now that you've mastered Queries, make more Relations

Relations

A Relation object represents one record pointing to another — such as the author (User) of a Comment, or the Post the comment belongs to.

Defining Relations

There's two steps to defining a relation:

1. A table column for the related record's ID

   tableSchema({
     name: 'comments',
     columns: [
       // ...
       { name: 'author_id', type: 'string' },
     ]
   }),
   

2. A @relation field defined on a Model class:

   import { relation } from '@nozbe/watermelondb/decorators'
   
   class Comment extends Model {
     // ...
     @relation('users', 'author_id') author
   }
   

   The first argument is the table name of the related record, and the second is the column name with an ID for the related record.

Relation API

In the example above, comment.author returns a Relation object.

Remember, WatermelonDB is a lazily-loaded database, so you don't get the related User record immediately, only when you explicitly fetch it

Observing

Most of the time, you connect Relations to Components by using observe() (the same as with Queries):

withObservables(['comment'], ({ comment }) => ({
  comment: comment.observe(),
  author: comment.author.observe(),
}))

The component will now have an author prop containing a User, and will re-render both when the user changes (e.g. comment's author changes its name), but also when a new author is assigned to the comment (if that was possible).

Fetching

To simply get the related record, use fetch. You might need it in Actions

const author = await comment.author.fetch()

// Shortcut syntax:
const author = await comment.author

Note: If the relation column (in this example, author_id) is marked as isOptional: true, fetch() might return null.

ID

If you only need the ID of a related record (e.g. to use in an URL or for the key= React prop), use id.

const authorId = comment.author.id

Assigning

Use set() to assign a new record to the relation

await commentsCollection.create(comment => {
  comment.author.set(someUser)
  // ...
})

Note: you can only do this in the .create() or .update() block.

You can also use set id if you only have the ID for the record to assign

await comment.update(() => {
  comment.author.id = userId
})

Advanced relations

immutableRelation

If you have a relation that cannot change (for example, a comment can't change its author), you can use @immutableRelation for extra protection and performance:

import { immutableRelation } from '@nozbe/watermelondb/decorators'

class Comment extends Model {
  // ...
  @immutableRelation('posts', 'post_id') post
  @immutableRelation('users', 'author_id') author
}

Many-To-Many Relation

If for instance, our app Posts can be authored by many Users and a user can author many Posts. We would create such a relation following these steps:-

1. Create a pivot schema and model that both the User model and Post model has association to; say PostAuthor
2. Create has_many association on both User and Post pointing to PostAuthor Model
3. Create belongs_to association on PostAuthor pointing to both User and Post
4. Retrieve all Posts for a user by defining a query that uses the pivot PostAuthor to infer the Posts that were authored by the User.

import {lazy } from '@nozbe/watermelondb/decorators'

class Post extends Model {
  static table = 'posts'
  static associations = {
    post_authors: { type: 'has_many', foreignKey: 'post_id' },
  }

  @lazy
  authors = this.collections
    .get('users')
    .query(Q.on('post_authors', 'post_id', this.id));
}

import { field } from '@nozbe/watermelondb/decorators'

class PostAuthor extends Model {
  static table = 'post_authors'
  static associations = {
    posts: { type: 'belongs_to', key: 'post_id' },
    users: { type: 'belongs_to', key: 'user_id' },
  }
  @field('post_id') postId
  @field('user_id') userId
}

import {lazy } from '@nozbe/watermelondb/decorators'

class User extends Model {
  static table = 'users'
  static associations = {
    post_authors: { type: 'has_many', foreignKey: 'user_id' },
  }

  @lazy
  posts = this.collections
    .get('posts')
    .query(Q.on('post_authors', 'user_id', this.id));

}

withObservables(['post'], ({ post }) => ({
  authors: post.authors.observe(),
}))

Next steps

➡️ Now the last step of this guide: define custom Actions

Actions

Although you can .create() and .update() records anywhere in your app, we recommend defining explicit Actions to encapsulate all ways to make changes.

Defining explicit Actions

An Action is a function that can modify the database (create, update, and delete records).

To define it, just add a method to a Model class marked with the @action decorator

import { action } from '@nozbe/watermelondb/decorators'

class Post extends Model {
  // ...

  @action async addComment(body, author) {
    return await this.collections.get('comments').create(comment => {
      comment.post.set(this)
      comment.author.set(author)
      comment.body = body
    })
  }
}

Note:

• Always mark actions as async and remember to await on .create() and .update()
• You can use this.collections to access Database.collections

Another example: updater action on Comment:

class Comment extends Model {
  // ...
  @field('is_spam') isSpam

  @action async markAsSpam() {
    await this.update(comment => {
      comment.isSpam = true
    })
  }
}

Now we can create a comment and immediately mark it as spam:

const comment = await post.addComment('Lorem ipsum', someUser)
await comment.markAsSpam()

Batch updates

Whenever you make more than one change (create, delete or update records) in an action, you should batch them.

It means that the app doesn't have to go back and forth with the database (sending one command, waiting for the response, then sending another), but instead sends multiple commands in one big batch. This is faster, safer, and can avoid subtle bugs in your app

Take an action that changes a Post into spam:

class Post extends Model {
  // ...
  @action async createSpam() {
    await this.update(post => {
      post.title = `7 ways to lose weight`
    })
    await this.collections.get('comments').create(comment => {
      comment.post.set(this)
      comment.body = "Don't forget to comment, like, and subscribe!"
    })
  }
}

Let's modify it to use batching:

class Post extends Model {
  // ...
  @action async createSpam() {
    await this.batch(
      this.prepareUpdate(post => {
        post.title = `7 ways to lose weight`
      }),
      this.collections.get('comments').prepareCreate(comment => {
        comment.post.set(this)
        comment.body = "Don't forget to comment, like, and subscribe!"
      })
    )
  }
}

Note:

• Call await this.batch in the Action (outside of actions, you can also call .batch() on the Database object)
• Pass the list of prepared operations as arguments:
  • Instead of calling await record.update(), pass record.prepareUpdate() — note lack of await
  • Instead of await collection.create(), use collection.prepareCreate()
  • Instead of await record.markAsDeleted(), use record.prepareMarkAsDeleted()
  • Instead of await record.destroyPermanently(), use record.prepareDestroyPermanently()
  • You can pass falsy values (null, undefined, false) to batch — they will simply be ignored.
  • You can also pass a single array argument instead of a list of arguments
• Otherwise, the API is the same!

Calling Actions from Actions

If you try to call an Action from another Action, you'll notice that it won't work. This is because while Action is running, no other Action can run simultaneously. To override this behavior, wrap the Action call in this.subAction:

class Comment extends Model {
  // ...

  @action async appendToPost() {
    const post = await this.post.fetch()
    // `appendToBody` is an `@action` on `Post`, so we call subAction to allow it
    await this.subAction(() => post.appendToBody(this.body))
  }
}

Delete action

When you delete, say, a Post, you generally want all Comments that belong to it to be deleted as well.

To do this, override markAsDeleted() (or destroyPermanently() if you don't sync) to explicitly delete all children as well.

class Post extends Model {
  static table = 'posts'
  static associations = {
    comments: { type: 'has_many', foreignKey: 'post_id' },
  }

  @children('comments') comments

  async markAsDeleted() {
    await this.comments.destroyAllPermanently()
    await super.markAsDeleted()
  }
}

Then to actually delete the post:

database.action(async () => {
  await post.markAsDeleted()
})

Note:

• Use Query.destroyAllPermanently() on all dependent @children you want to delete
• Remember to call super.markAsDeleted — at the end of the method!

Inline actions

If you want to call a number of write operations outside of a Model action, do it like so:

const newPost = await database.action(async action => {
  // Note: function passed to `database.action()` MUST be asynchronous
  const posts = database.collections.get('posts')
  const post = await posts.create( /* configure Post here */ )

  // Note: to call an action from an inline action, call `action.subAction`:
  await action.subAction(() => post.markAsPromoted())

  // Note: Value returned from the wrapped function will be returned to `database.action` caller
  return post
})

Advanced: Why actions are necessary?

WatermelonDB is highly asynchronous, which is a BIG challange in terms of achieving consistent data. Read this only if you are curious:

Consider a function markCommentsAsSpam that fetches a list of comments on a post, and then marks them all as spam. The two operations (fetching, and then updating) are asynchronous, and some other operation that modifies the database could run in between. And it could just happen to be a function that adds a new comment on this post. Even though the function completes successfully, it wasn't actually successful at its job.

This example is trivial. But others may be far more dangerous. If a function fetches a record to perform an update on, this very record could be deleted midway through, making the action fail (and potentially causing the app to crash, if not handled properly). Or a function could have invariants determining whether the user is allowed to perform an action, that would be invalidated during action's execution. Or, in a collaborative app where access permissions are represented by another object, parallel execution of different actions could cause those access relations to be left in an inconsistent state.

The worst part is that analyzing all possible interactions for dangers is very hard, and having sync that runs automatically makes them very likely.

Solution? Group together related reads and writes together in an Action, enforce that writes MUST occur in an Action, and only allow one Action to run at the time. This way, it's guaranteed that in an action, you're looking at a consistent view of the world. On the other hand, most reads are safe to perform without grouping them. If you suspect they're not, you can also wrap them in an Action.

Next steps

➡️ Now that you've mastered all basics of Watermelon, go create some powerful apps — or keep reading advanced guides

Advanced guides

Advanced guides for using WatermelonDB

Migrations

Schema migrations is the mechanism by which you can add new tables and columns to the database in a backward-compatible way.

Without migrations, if a user of your app upgrades from one version to another, their local database will be cleared at launch, and they will lose all their data.

⚠️ Always use migrations!

Migrations setup

1. Add a new file for migrations:

   // app/model/migrations.js
   
   import { schemaMigrations } from '@nozbe/watermelondb/Schema/migrations'
   
   export default schemaMigrations({
     migrations: [
       // We'll add migration definitions here later
     ],
   })
   

2. Hook up migrations to the Database adapter setup:

   // index.js
   import migrations from 'model/migrations'
   
   const adapter = new SQLiteAdapter({
     schema: mySchema,
     migrations,
   })
   

Migrations workflow

When you make schema changes when you use migrations, be sure to do this in this specific order, to minimize the likelihood of making an error.

Step 1: Add a new migration

First, define the migration - that is, define the change that occurs between two versions of schema (such as adding a new table, or a new table column).

Don't change the schema file yet!

// app/model/migrations.js

import { schemaMigrations, createTable } from '@nozbe/watermelondb/Schema/migrations'

export default schemaMigrations({
  migrations: [
    {
      // ⚠️ Set this to a number one larger than the current schema version
      toVersion: 2,
      steps: [
        // See "Migrations API" for more details
        createTable({
          name: 'comments',
          columns: [
            { name: 'post_id', type: 'string', isIndexed: true },
            { name: 'body', type: 'string' },
          ],
        }),
      ],
    },
  ],
})

Refresh your simulator/browser. You should see this error:

Migrations can't be newer than schema. Schema is version 1 and migrations cover range from 1 to 2

If so, good, move to the next step!

But you might also see an error like "Missing table name in schema", which means you made an error in defining migrations. See "Migrations API" below for details.

Step 2: Make matching changes in schema

Now it's time to make the actual changes to the schema file — add the same tables or columns as in your migration definition

⚠️ Please double and triple check that your changes to schema match exactly the change you defined in the migration. Otherwise you risk that the app will work when the user migrates, but will fail if it's a fresh install — or vice versa.

⚠️ Don't change the schema version yet

// model/schema.js

export default appSchema({
  version: 1,
  tables: [
    // This is our new table!
    tableSchema({
      name: 'comments',
      columns: [
        { name: 'post_id', type: 'string', isIndexed: true },
        { name: 'body', type: 'string' },
      ],
    }),
    // ...
  ]
})

Refresh the simulator. You should again see the same "Migrations can't be newer than schema" error. If you see a different error, you made a syntax error.

Step 3: Bump schema version

Now that we made matching changes in the schema (source of truth about tables and columns) and migrations (the change in tables and columns), it's time to commit the change by bumping the version:

// model/schema.js

export default appSchema({
  version: 2,
  tables: [
    // ...
  ]
})

If you refresh again, your app should show up without issues — but now you can use the new tables/columns

Step 4: Test your migrations

Before shipping a new version of the app, please check that your database changes are all compatible:

1. Migrations test: Install the previous version of your app, then update to the version you're about to ship, and make sure it still works
2. Fresh schema install test: Remove the app, and then install the new version of the app, and make sure it works

Why is this order important

It's simply because React Native simulator (and often React web projects) are configured to automatically refresh when you save a file. You don't want to database to accidentally migrate (upgrade) with changes that have a mistake, or changes you haven't yet completed making. By making migrations first, and bumping version last, you can double check you haven't made a mistake.

Migrations API

Each migration must migrate to a version one above the previous migration, and have multiple steps (such as adding a new table, or new columns). Larger example:

schemaMigrations({
  migrations: [
    {
      toVersion: 3,
      steps: [
        createTable({
          name: 'comments',
          columns: [
            { name: 'post_id', type: 'string', isIndexed: true },
            { name: 'body', type: 'string' },
          ],
        }),
        addColumns({
          table: 'posts',
          columns: [
            { name: 'subtitle', type: 'string', isOptional: true },
            { name: 'is_pinned', type: 'boolean' },
          ],
        }),
      ],
    },
    {
      toVersion: 2,
      steps: [
        // ...
      ],
    },
  ],
})

Migration steps:

• createTable({ name: 'table_name', columns: [ ... ] }) - same API as tableSchema()
• addColumns({ table: 'table_name', columns: [ ... ] }) - you can add one or multiple columns to an existing table. The columns table has the same format as in schema definitions
• Other types of migrations (e.g. deleting or renaming tables and columns) are not yet implemented. See migrations/index.js. Please contribute!

Database reseting and other edge cases

1. When you're not using migrations, the database will reset (delete all its contents) whenever you change the schema version.
2. If the migration fails, the database will fail to initialize, and will roll back to previous version. This is unlikely, but could happen if you, for example, create a migration that tries to create the same table twice. The reason why the database will fail instead of reset is to avoid losing user data (also it's less confusing in development). You can notice the problem, fix the migration, and ship it again without data loss.
3. When database in the running app has newer database version than the schema version defined in code, the database will reset (clear its contents). This is useful in development
4. If there's no available migrations path (e.g. user has app with database version 4, but oldest migration is from version 10 to 11), the database will reset.

Rolling back changes

There's no automatic "rollback" feature in Watermelon. If you make a mistake in migrations during development, roll back in this order:

1. Comment out any changes made to schema.js
2. Comment out any changes made to migrations.js
3. Decrement schema version number (bring back the original number)

After refreshing app, the database should reset to previous state. Now you can correct your mistake and apply changes again (please do it in order described in "Migrations workflow").

Synchronization

WatermelonDB has been designed from scratch to be able to seamlessly synchronize with a remote database (and, therefore, keep multiple copies of data synced with each other).

Note that Watermelon is only a local database — you need to bring your own backend. What Watermelon provides are:

• Synchronization primitives — information about which records were created, updated, or deleted locally since the last sync — and which columns exactly were modified. You can build your own custom sync engine using those primitives
• Built-in sync adapter — You can use the sync engine Watermelon provides out of the box, and you only need to provide two API endpoints on your backend that conform to Watermelon sync protocol

Using synchronize() in your app

To synchronize, you need to pass two functions, pullChanges and pushChanges that talk to your backend and are compatible with Watermelon Sync Protocol. The frontend code will look something like this:

import { synchronize } from '@nozbe/watermelondb/sync'

async function mySync() {
  await synchronize({
    database,
    pullChanges: async ({ lastPulledAt, schemaVersion, migration }) => {
      const response = await fetch(`https://my.backend/sync`, {
        body: JSON.stringify({ lastPulledAt, schemaVersion, migration })
      })
      if (!response.ok) {
        throw new Error(await response.text())
      }

      const { changes, timestamp } = await response.json()
      return { changes, timestamp }
    },
    pushChanges: async ({ changes, lastPulledAt }) => {
      const response = await fetch(`https://my.backend/sync?last_pulled_at=${lastPulledAt}`, {
        method: 'POST',
        body: JSON.stringify(changes)
      })
      if (!response.ok) {
        throw new Error(await response.text())
      }
    },
    migrationsEnabledAtVersion: 1,
  })
}

Troubleshooting

⚠️ Note about a React Native / UglifyES bug. When you import Watermelon Sync, your app might fail to compile in release mode. To fix this, configure Metro bundler to use Terser instead of UglifyES. Run:

yarn add metro-minify-terser

Then, update metro.config.js:

module.exports = {
  // ...
  transformer: {
    // ...
    minifierPath: 'metro-minify-terser',
  },
}

You might also need to switch to Terser in Webpack if you use Watermelon for web.

Implementing pullChanges()

Watermelon will call this function to ask for changes that happened on the server since the last pull.

Arguments:

• lastPulledAt is a timestamp for the last time client pulled changes from server (or null if first sync)
• schemaVersion is the current schema version of the local database
• migration is an object representing schema changes since last sync (or null if up to date or not supported)

This function should fetch from the server the list of ALL changes in all collections since lastPulledAt.

1. You MUST pass an async function or return a Promise that eventually resolves or rejects
2. You MUST pass lastPulledAt, schemaVersion, and migration to an endpoint that conforms to Watermelon Sync Protocol
3. You MUST return a promise resolving to an object of this shape (your backend SHOULD return this shape already):

   {
     changes: { ... }, // valid changes object
     timestamp: 100000, // integer with *server's* current time
   }
   

4. You MUST NOT store the object returned in pullChanges(). If you need to do any processing on it, do it before returning the object. Watermelon treats this object as "consumable" and can mutate it (for performance reasons)

Implementing pushChanges()

Watermelon will call this function with a list of changes that happened locally since the last push so you can post it to your backend.

Arguments passed:

{
  changes: { ... }, // valid changes object
  lastPulledAt: 10000, // the timestamp of the last successful pull (timestamp returned in pullChanges)
}

1. You MUST pass changes and lastPulledAt to a push sync endpoint conforming to Watermelon Sync Protocol
2. You MUST pass an async function or return a Promise from pushChanges()
3. pushChanges() MUST resolve after and only after the backend confirms it successfully received local changes
4. pushChanges() MUST reject if backend failed to apply local changes
5. You MUST NOT resolve sync prematurely or in case of backend failure
6. You MUST NOT mutate or store arguments passed to pushChanges(). If you need to do any processing on it, do it before returning the object. Watermelon treats this object as "consumable" and can mutate it (for performance reasons)

General information and tips

1. You MUST NOT connect to backend endpoints you don't control using synchronize(). WatermelonDB assumes pullChanges/pushChanges are friendly and correct and does not guarantee secure behavior if data returned is malformed.
2. You SHOULD NOT call synchronize() while synchronization is already in progress (it will safely abort)
3. You MUST NOT reset local database while synchronization is in progress (push to server will be safely aborted, but consistency of the local database may be compromised)
4. You SHOULD wrap synchronize() in a "retry once" block - if sync fails, try again once. This will resolve push failures due to server-side conflicts by pulling once again before pushing.
5. You can use database.withChangesForTables to detect when local changes occured to call sync. If you do this, you should debounce (or throttle) this signal to avoid calling synchronize() too often.

Adopting Migration Syncs

For Watermelon Sync to maintain consistency after migrations, you must support Migration Syncs (introduced in WatermelonDB v0.17). This allows Watermelon to request from backend the tables and columns it needs to have all the data.

1. For new apps, pass {migrationsEnabledAtVersion: 1} to synchronize() (or the first schema version that shipped / the oldest schema version from which it's possible to migrate to the current version)
2. To enable migration syncs, the database MUST be configured with migrations spec (even if it's empty)
3. For existing apps, set migrationsEnabledAtVersion to the current schema version before making any schema changes. In other words, this version should be the last schema version BEFORE the first migration that should support migration syncs.
4. Note that for apps that shipped before WatermelonDB v0.17, it's not possible to determine what was the last schema version at which the sync happened. migrationsEnabledAtVersion is used as a placeholder in this case. It's not possible to guarantee that all necessary tables and columns will be requested. (If user logged in when schema version was lower than migrationsEnabledAtVersion, tables or columns were later added, and new records in those tables/changes in those columns occured on the server before user updated to an app version that has them, those records won't sync). To work around this, you may specify migrationsEnabledAtVersion to be the oldest schema version from which it's possible to migrate to the current version. However, this means that users, after updating to an app version that supports Migration Syncs, will request from the server all the records in new tables. This may be unacceptably inefficient.
5. WatermelonDB >=0.17 will note the schema version at which the user logged in, even if migrations are not enabled, so it's possible for app to request from backend changes from schema version lower than migrationsEnabledAtVersion
6. You MUST NOT delete old migrations, otherwise it's possible that the app is permanently unable to sync.

Adding logging to your sync

You can add basic sync logs to the sync process by passing an empty object to synchronize(). Sync will then mutate the object, populating it with diagnostic information (start/finish time, resolved conflicts, and more):

const log = {}
await synchronize({
database,
log,
...
})
console.log(log.startedAt)
console.log(log.finishedAt)

⚠️ Remember to act responsibly with logs, since they might contain your user's private information. Don't display, save, or send the log unless you censor the log. Example logger and censor code you can use.

Additional synchronize() flags

• _unsafeBatchPerCollection: boolean - if true, changes will be saved to the database in multiple batches. This is unsafe and breaks transactionality, however may be required for very large syncs due to memory issues
• sendCreatedAsUpdated: boolean - if your backend can't differentiate between created and updated records, set this to true to supress warnings. Sync will still work well, however error reporting, and some edge cases will not be handled as well.

Implementing your Sync backend

Understanding changes objects

Synchronized changes (received by the app in pullChanges and sent to the backend in pushChanges) are represented as an object with raw records. Those only use raw table and column names, and raw values (strings/numbers/booleans) — the same as in Schema.

Deleted objects are always only represented by their IDs.

Example:

{
  projects: {
    created: [
      { id: 'aaaa', name: 'Foo', is_favorite: true },
      { id: 'bbbb', name: 'Bar', is_favorite: false },
    ],
    updated: [
      { id: 'ccc', name: 'Baz', is_favorite: true },
    ],
    deleted: ['ddd'],
  },
  tasks: {
    created: [],
    updated: [
      { id: 'tttt', name: 'Buy eggs' },
    ],
    deleted: [],
  },
  ...
}

Valid changes objects MUST conform to this shape:

Changes = {
  [table_name: string]: {
    created: RawRecord[],
    updated: RawRecord[],
    deleted: string[],
  }
}

Implementing pull endpoint

Expected parameters:

{
  lastPulledAt: Timestamp,
  schemaVersion: int,
  migration: null | { from: int, tables: string[], columns: { table: string, columns: string[] }[] }
}

Expected response:

{ changes: Changes, timestamp: Timestamp }

1. The pull endpoint SHOULD take parameters and return a response matching the shape specified above. This shape MAY be different if negotiated with the frontend (however, frontend-side pullChanges() MUST conform to this)
2. The pull endpoint MUST return all record changes in all collections since lastPulledAt, specifically:
   • all records that were created on the server since lastPulledAt
   • all records that were updated on the server since lastPulledAt
   • IDs of all records that were deleted on the server since lastPulledAt
3. If lastPulledAt is null or 0, you MUST return all accessible records (first sync)
4. The timestamp returned by the server MUST be a value that, if passed again to pullChanges() as lastPulledAt, will return all changes that happened since this moment.
5. The pull endpoint MUST provide a consistent view of changes since lastPulledAt
   • You should perform all queries synchronously or in a write lock to ensure that returned changes are consistent
   • You should also mark the current server time synchronously with the queries
   • This is to ensure that no changes are made to the database while you're fetching changes (otherwise some records would never be returned in a pull query)
   • If it's absolutely not possible to do so, and you have to query each collection separately, be sure to return a lastPulledAt timestamp marked BEFORE querying starts. You still risk inconsistent responses (that may break app's consistency assumptions), but the next pull will fetch whatever changes occured during previous pull.
   • An alternative solution is to check for the newest change before and after all queries are made, and if there's been a change during the pull, return an error code, or retry.
6. If migration is not null, you MUST include records needed to get a consistent view after a local database migration
   • Specifically, you MUST include all records in tables that were added to the local database between the last user sync and schemaVersion
   • For all columns that were added to the local app database between the last sync and schemaVersion, you MUST include all records for which the added column has a value other than the default value (0, '', false, or null depending on column type and nullability)
   • You can determine what schema changes were made to the local app in two ways:
     • You can compare migration.from (local schema version at the time of the last sync) and schemaVersion (current local schema version). This requires you to negotiate with the frontend what schema changes are made at which schema versions, but gives you more control
     • Or you can ignore migration.from and only look at migration.tables (which indicates which tables were added to the local database since the last sync) and migration.columns (which indicates which columns were added to the local database to which tables since last sync).
     • If you use migration.tables and migration.columns, you MUST whitelist values a client can request. Take care not to leak any internal fields to the client.
7. Returned raw records MUST match your app's Schema
8. Returned raw records MUST NOT not contain special _status, _changed fields.
9. Returned raw records MAY contain fields (columns) that are not yet present in the local app (at schemaVersion -- but added in a later version). They will be safely ignored.
10. Returned raw records MUST NOT contain arbitrary column names, as they may be unsafe (e.g. __proto__ or constructor). You should whitelist acceptable column names.
11. Returned record IDs MUST only contain safe characters
    • Default WatermelonDB IDs conform to /^[a-zA-Z0-9]{16}$/
    • _-. are also allowed if you override default ID generator, but '"\/$ are unsafe
12. Changes SHOULD NOT contain collections that are not yet present in the local app (at schemaVersion). They will, however, be safely ignored.
    • NOTE: This is true for WatermelonDB v0.17 and above. If you support clients using earlier versions, you MUST NOT return collections not known by them.
13. Changes MUST NOT contain collections with arbitrary names, as they may be unsafe. You should whitelist acceptable collection names.

Implementing push endpoint

1. The push endpoint MUST apply local changes (passed as a changes object) to the database. Specifically:
   • create new records as specified by the changes object
   • update existing records as specified by the changes object
   • delete records by the specified IDs
2. If the changes object contains a new record with an ID that already exists, you MUST update it, and MUST NOT return an error code.
   • (This happens if previous push succeeded on the backend, but not on frontend)
3. If the changes object contains an update to a record that does not exist, then:
   • If you can determine that this record no longer exists because it was deleted, you SHOULD return an error code (to force frontend to pull the information about this deleted ID)
   • Otherwise, you MUST create it, and MUST NOT return an error code. (This scenario should not happen, but in case of frontend or backend bugs, it would keep sync from ever succeeding.)
4. If the changes object contains a record to delete that doesn't exist, you MUST ignore it and MUST NOT return an error code
   • (This may happen if previous push succeeded on the backend, but not on frontend, or if another user deleted this record in between user's pull and push calls)
5. If the changes object contains a record that has been modified on the server after lastPulledAt, you MUST abort push and return an error code
   • This scenario means that there's a conflict, and record was updated remotely between user's pull and push calls. Returning an error forces frontend to call pull endpoint again to resolve the conflict
6. If application of all local changes succeeds, the endpoint MUST return a success status code.
7. The push endpoint MUST be fully transactional. If there is an error, all local changes MUST be reverted, and en error code MUST be returned.
8. You MUST ignore _status and _changed fields contained in records in changes object
9. You SHOULD validate data passed to the endpoint. In particular, collection and column names ought to be whitelisted, as well as ID format — and of course any application-specific invariants, such as permissions to access and modify records
10. You SHOULD sanitize record fields passed to the endpoint. If there's something slightly wrong with the contents (but not shape) of the data (e.g. user.role should be owner, admin, or member, but user sent empty string or abcdef), you SHOULD NOT send an error code. Instead, prefer to "fix" errors (sanitize to correct format).
    • Rationale: Synchronization should be reliable, and should not fail other than transiently, or for serious programming errors. Otherwise, the user will have a permanently unsyncable app, and may have to log out/delete it and lose unsynced data. You don't want a bug 5 versions ago to create a persistently failing sync.
11. You SHOULD delete all descendants of deleted records
    • Frontend should ask the push endpoint to do so as well, but if it's buggy, you may end up with permanent orphans

Tips on implementing server-side changes tracking

If you're wondering how to actually implement consistent pulling of all changes since the last pull, or how to detect that a record being pushed by the user changed after lastPulledAt, here's what we recommend:

• Add a last_modified field to all your server database tables, and bump it to NOW() every time you create or update a record.
• This way, when you want to get all changes since lastPulledAt, you query records whose last_modified > lastPulledAt.
• The timestamp should be at least millisecond resolution, and you should add (for extra safety) a MySQL/PostgreSQL procedure that will ensure last_modified uniqueness and monotonicity
  • Specificaly, check that there is no record with a last_modified equal to or greater than NOW(), and if there is, increment the new timestamp by 1 (or however much you need to ensure it's the greatest number)
  • An example of this for PostgreSQL can be found in Kinto
  • This protects against weird edge cases - such as records being lost due to server clock time changes (NTP time sync, leap seconds, etc.)
• Of course, remember to ignore last_modified from the user if you do it this way.
• An alternative to using timestamps is to use an auto-incrementing counter sequence, but you must ensure that this sequence is consistent across all collections. You also leak to users the amount of traffic to your sync server (number of changes in the sequence)
• To distinguish between created and updated records, you can also store server-side server_created_at timestamp (if it's greater than last_pulled_at supplied to sync, then record is to be created on client, if less than — client already has it and it is to be updated on client). Note that this timestamp must be consistent with last_modified — and you must not use client-created created_at field, since you can never trust local timestamps.
  • Alternatively, you can send all non-deleted records as all updated and Watermelon will do the right thing in 99% of cases (you will be slightly less protected against weird edge cases — treatment of locally deleted records is different). If you do this, pass sendCreatedAsUpdated: true to synchronize() to supress warnings about records to be updated not existing locally.
• You do need to implement a mechanism to track when records were deleted on the server, otherwise you wouldn't know to push them
  • One possible implementation is to not fully delete records, but mark them as DELETED=true
  • Or, you can have a deleted_xxx table with just the record ID and timestamp (consistent with last_modified)
  • Or, you can treat it the same way as "revoked permissions"
• If you have a collaborative app with any sort of permissions, you also need to track granting and revoking of permissions the same way as changes to records
  • If permission to access records has been granted, the pull endpoint must add those records to created
  • If permission to access records has been revoked, the pull endpoint must add those records to deleted
  • Remember to also return all descendants of a record in those cases

Local vs Remote IDs

WatermelonDB has been designed with the assumption that there is no difference between Local IDs (IDs of records and their relations in a WatermelonDB database) and Remote IDs (IDs on the backend server). So a local app can create new records, generating their IDs, and the backend server will use this ID as the true ID. This greatly simplifies synchronization, as you don't have to replace local with remote IDs on the record and all records that point to it.

We highly recommend that you adopt this practice.

Some people are skeptical about this approach due to conflicts, since backend can guarantee unique IDs, and the local app can't. However, in practice, a standard Watermelon ID has 8,000,000,000,000,000,000,000,000 possible combinations. That's enough entropy to make conflicts extremely unlikely. At Nozbe, we've done it this way at scale for more than a decade, and not once did we encounter a genuine ID conflict or had other issues due to this approach.

Using the birthday problem, we can calculate that for 36^16 possible IDs, if your system grows to a billion records, the probability of a single conflict is 6e-8. At 100B records, the probability grows to 0.06%. But if you grow to that many records, you're probably a very rich company and can start worrying about things like this then.

If you absolutely can't adopt this practice, there's a number of production apps using WatermelonDB that keep local and remote IDs separate — however, more work is required this way. Search Issues to find discussions about this topic — and consider contributing to WatermelonDB to make managing separate local IDs easier for everyone!

Existing backend implementations for WatermelonDB

Note that those are not maintained by WatermelonDB, and we make no endorsements about quality of these projects:

• How to Build WatermelonDB Sync Backend in Elixir
• Firemelon
• Did you make one? Please contribute a link!

Current Sync limitations

1. If a record being pushed changes between pull and push, push will just fail. It would be better if it failed with a list of conflicts, so that synchronize() can automatically respond. Alternatively, sync could only send changed fields and server could automatically always just apply those changed fields to the server version (since that's what per-column client-wins resolver will do anyway)
2. During next sync pull, changes we've just pushed will be pulled again, which is unnecessary. It would be better if server, during push, also pulled local changes since lastPulledAt and responded with NEW timestamp to be treated as lastPulledAt.
3. It shouldn't be necessary to push the whole updated record — just changed fields + ID should be enough

Note: That might conflict with "If client wants to update a record that doesn’t exist, create it"

You don't like these limitations? Good, neither do we! Please contribute - we'll give you guidance.

Contributing

1. If you implement Watermelon sync but found this guide confusing, please contribute improvements!
2. Please help out with solving the current limitations!
3. If you write server-side code made to be compatible with Watermelon, especially for popular platforms (Node, Ruby on Rails, Kinto, etc.) - please open source it and let us know! This would dramatically simplify implementing sync for people
4. If you find Watermelon sync bugs, please report the issue! And if possible, write regression tests to make sure it never happens again

Sync primitives and implementing your own sync entirely from scratch

See: Sync implementation details

Create/Update tracking

You can add per-table support for create/update tracking. When you do this, the Model will have information about when it was created, and when it was last updated.

When to use this

Use create tracking:

• When you display to the user when a thing (e.g. a Post, Comment, Task) was created
• If you sort created items chronologically (Note that Record IDs are random strings, not auto-incrementing integers, so you need create tracking to sort chronologically)

Use update tracking:

• When you display to the user when a thing (e.g. a Post) was modified

Note: you don't have to enable both create and update tracking. You can do either, both, or none.

How to do this

Step 1: Add to the schema:

tableSchema({
  name: 'posts',
  columns: [
    // other columns
    { name: 'created_at', type: 'number' },
    { name: 'updated_at', type: 'number' },
  ]
}),

Step 2: Add this to the Model definition:

import { date, readonly } from '@nozbe/watermelondb/decorators'

class Post extends Model {
  // ...
  @readonly @date('created_at') createdAt
  @readonly @date('updated_at') updatedAt
}

Again, you can add just created_at column and field if you don't need update tracking.

How this behaves

If you have the magic createdAt field defined on the Model, the current timestamp will be set when you first call collection.create() or collection.prepareCreate(). It will never be modified again.

If the magic updatedAt field is also defined, then after creation, model.updatedAt will have the same value as model.createdAt. Then every time you call model.update() or model.prepareUpdate(), updatedAt will be changed to the current timestamp.

Advanced Fields

@text

You can use @text instead of @field to enable user text sanitization. When setting a new value on a @text field, excess whitespace will be trimmed from both ends from the string.

import { text } from '@nozbe/watermelondb/decorators'

class Post extends Model {
  // ...
  @text('body') body
}

@json

If you have a lot of metadata about a record (say, an object with many keys, or an array of values), you can use a @json field to contain that information in a single string column (serialized to JSON) instead of adding multiple columns or a relation to another table.

⚠️ This is an advanced feature that comes with downsides — make sure you really need it

First, add a string column to the schema:

tableSchema({
  name: 'comments',
  columns: [
    { name: 'reactions', type: 'string' }, // You can add isOptional: true, if appropriate
  ],
})

Then in the Model definition:

import { json } from '@nozbe/watermelondb/decorators'

class Comment extends Model {
  // ...
  @json('reactions', sanitizeReactions) reactions
}

Now you can set complex JSON values to a field:

comment.update(() => {
  comment.reactions = ['up', 'down', 'down']
})

As the second argument, pass a sanitizer function. This is a function that receives whatever JSON.parse() returns for the serialized JSON, and returns whatever type you expect in your app. In other words, it turns raw, dirty, untrusted data (that might be missing, or malformed by a bug in previous version of the app) into trusted format.

The sanitizer might also receive null if the column is nullable, or undefined if the field doesn't contain valid JSON.

For example, if you need the field to be an array of strings, you can ensure it like so:

const sanitizeReactions = rawReactions => {
  return Array.isArray(rawReactions) ? rawReactions.map(String) : []
}

If you don't want to sanitize JSON, pass an identity function:

const sanitizeReactions = json => json

The sanitizer function takes an optional second argument, which is a reference to the model. This is useful is your sanitization logic depends on the other fields in the model.

Warning about JSON fields:

JSON fields go against relational, lazy nature of Watermelon, because you can't query or count by the contents of JSON fields. If you need or might need in the future to query records by some piece of data, don't use JSON.

Only use JSON fields when you need the flexibility of complex freeform data, or the speed of having metadata without querying another table, and you are sure that you won't need to query by those metadata.

@nochange

For extra protection, you can mark fields as @nochange to ensure they can't be modified. Always put @nochange before @field / @date / @text

import { field, nochange } from '@nozbe/watermelondb/decorators'

class User extends Model {
  // ...
  @nochange @field('is_owner') isOwner
}

user.isOwner can only be set in the collection.create() block, but will throw an error if you try to set a new value in user.update() block.

@readonly

Similar to @nochange, you can use the @readonly decorator to ensure a field cannot be set at all. Use this for create/update tracking, but it might also be useful if you use Watermelon with a Sync engine and a field can only be set by the server.

Custom observable fields

You're in advanced RxJS territory now! You have been warned.

Say, you have a Post model that has many Comments. And a Post is considered to be "popular" if it has more than 10 comments.

You can add a "popular" badge to a Post component in two ways.

One is to simply observe how many comments there are in the component:

const enhance = withObservables(['post'], ({ post }) => ({
  post: post.observe(),
  commentCount: post.comments.observeCount()
}))

And in the render method, if props.commentCount > 10, show the badge.

Another way is to define an observable property on the Model layer, like so:

import { distinctUntilChanged, map as map$ } from 'rxjs/operators'
import { lazy } from '@nozbe/watermelondb/decorators'

class Post extends Model {
  @lazy isPopular = this.comments.observeCount().pipe(
    map$(comments => comments > 10),
    distinctUntilChanged()
  )
}

And then you can directly connect this to the component:

const enhance = withObservables(['post'], ({ post }) => ({
  isPopular: post.isPopular,
}))

props.isPopular will reflect whether or not the Post is popular. Note that this is fully observable, i.e. if the number of comments rises above/falls below the popularity threshold, the component will re-render. Let's break it down:

• this.comments.observeCount() - take the Observable number of comments
• map$(comments => comments > 10) - transform this into an Observable of boolean (popular or not)
• distinctUntilChanged() - this is so that if the comment count changes, but the popularity doesn't (it's still below/above 10), components won't be unnecessarily re-rendered
• @lazy - also for performance (we only define this Observable once, so we can re-use it for free)

Let's make this example more complicated. Say the post is always popular if it's marked as starred. So if post.isStarred, then we don't have to do unnecessary work of fetching comment count:

import { of as of$ } from 'rxjs/observable/of'
import { distinctUntilChanged, map as map$ } from 'rxjs/operators'
import { lazy } from '@nozbe/watermelondb/decorators'

class Post extends Model {
  @lazy isPopular = this.observe().pipe(
    distinctUntilKeyChanged('isStarred'),
    switchMap(post =>
      post.isStarred ?
        of$(true) :
        this.comments.observeCount().pipe(map$(comments => comments > 10))
    ),
    distinctUntilChanged(),
  )
}

• this.observe() - if the Post changes, it might change its popularity status, so we observe it
• this.comments.observeCount().pipe(map$(comments => comments > 10)) - this part is the same, but we only observe it if the post is starred
• switchMap(post => post.isStarred ? of$(true) : ...) - if the post is starred, we just return an Observable that emits true and never changes.
• distinctUntilKeyChanged('isStarred') - for performance, so that we don't re-subscribe to comment count Observable if the post changes (only if the isStarred field changes)
• distinctUntilChanged() - again, don't emit new values, if popularity doesn't change

Watermelon ❤️ Flow

Watermelon was developed with Flow in mind.

If you're a Flow user yourself (and we highly recommend it!), here's some things you need to keep in mind:

Setup

Add this to your .flowconfig file so that Flow can see Watermelon's types.

[options]

module.name_mapper='^@nozbe/watermelondb\(.*\)$' -> '<PROJECT_ROOT>/node_modules/@nozbe/watermelondb/src\1'

Note that this won't work if you put the entire node_modules/ folder under the [ignore] section. In that case, change it to only ignore the specific node modules that throw errors in your app, so that Flow can scan Watermelon files.

Tables and columns

Table and column names are opaque types in Flow.

So if you try to use simple strings, like so:

class Comment extends Model {
  static table = 'comments'

  @text('body') body
}

You'll get errors, because you're passing 'comments' (a string) where TableName<Comment> is expected, and 'body' (again, a string) where ColumnName is expected.

When using Watermelon with Flow, you must pre-define all your table and column names in one place, then only use those symbols (and not strings) in all other places.

We recommend defining symbols like this:

// File: model/schema.js
// @flow

import { tableName, columnName, type TableName, appSchema, tableSchema } from '@nozbe/watermelondb'
import type Comment from './Comment.js'

export const Tables = {
  comments: (tableName('comments'): TableName<Comment>),
  // ...
}

export const Columns = {
  comments: {
    body: columnName('body'),
    // ...
  }
}

export const appSchema = appSchema({
  version: 1,
  tables: [
    tableSchema({
      name: Tables.comments,
      columns: [
        { name: Columns.comments.body, type: 'string' },
      ],
    }),
    // ...
  ]
})

And then using them like so:

// File: model/Comment.js
// @flow

import { Model } from '@nozbe/watermelondb'
import { text } from '@nozbe/watermelondb/decorators'

import { Tables, Columns } from './schema.js'

const Column = Columns.comments

export default class Comment extends Model {
  static table = Tables.comments

  @text(Column.body) body: string
}

But isn't that a lot of boilerplate?

Yes, it looks more boilerplate'y than the non-Flow examples, however:

• you're protected from typos — strings are defined once
• easier refactoring — you only change column name in one place
• no orphan columns or tables — no way to accidentally refer to a column or table that was removed from the schema
• TableName is typed with the model class it refers to, which allows Flow to find other mistakes in your code

In general, we find that untyped string constants lead to bugs, and defining typed constants is a good practice.

associations

When using Flow, you define model associations like this:

import { Model, associations } from '@nozbe/watermelondb'
import { Tables, Columns } from './schema.js'

const Column = Columns.posts

class Post extends Model {
  static table = Tables.posts
  static associations = associations(
    [Tables.comments, { type: 'has_many', foreignKey: Columns.comments.postId }],
    [Tables.users, { type: 'belongs_to', key: Column.authorId }],
  )
}

Common types

Many types are tagged with the model class the type refers to:

TableName<Post> // a table name referring to posts
Collection<Post> // the Collection for posts
Relation<Comment> // a relation that can fetch a Comment
Relation<?Comment> // a relation that can fetch a Comment or `null`
Query<Comment> // a query that can fetch many Comments

Always mark the type of model fields. Remember to include ? if the underlying table column is optional. Flow can't check if model fields match the schema or if they match the decorator's signature.

@text(Column.body) body: string
@date(Column.createdAt) createdAt: Date
@date(Column.archivedAt) archivedAt: ?Date

If you need to refer to an ID of a record, always use the RecordId type alias, not string (they're the same, but the former is self-documenting).

If you ever access the record's raw data (DON'T do that unless you really know what you're doing), use DirtyRaw to refer to raw data from external sources (database, server), and RawRecord after it was passed through sanitizedRaw.

Local storage

WatermelonDB has a simple key/value store, similar to localStorage:

// setting a value
await database.adapter.setLocal("user_id", "abcdef")

// retrieving a value
const userId = await database.adapter.getLocal("user_id") // string or null if no value for this key

// removing a value
await database.adapter.removeLocal("user_id")

When to use it. For things like the ID of the logged-in user, or the route to the last-viewed screen in the app. You should generally avoid it and stick to standard Watermelon records.

This is a low-level API. You can't do things like observe changes of a value over time. If you need that, just use standard WatermelonDB records. Also, you can only store strings. You can build your own abstraction that (de)serializes those values to/from JSON.

What to be aware of. DO NOT let the local storage key be a user-supplied value. Only allow predefined/whitelisted keys.

Why not use localStorage/AsyncStorage? Because this way, you have only one source of truth — one database that, say, stores the logged-in user ID and the information about all users. So there's a lower risk that the two sets of values get out of sync.

Performance

Performance tips — TODO

Dig deeper into WatermelonDB

Details about how Watermelon works, how to hack and contribute

📺 Digging deeper into WatermelonDB — more architectural info about caching, observation, and sync

Architecture

Base objects

Database is the root object of Watermelon. It owns:

• a DatabaseAdapter
• a map of Collections

DatabaseAdapter connects Watermelon's reactive world to low-level imperative world of databases. See Adapters.

Collection manages all records of a given kind:

• it has a cache of records already fetched from the database (RecordCache)
• it has the public API to find, query and create existing records
• it implements fetch/update/delete operations on records

Model is an instance of a collection record. A model class describes a kind of a record. Model is the base class for your concrete models (e.g. Post, Comment, Task):

• it describes the specific instance - id + all custom fields and actions
• it has public API to update, markAsDeleted and destroyPermanently
• implements record-level observation observe()
• static fields describe base information about a model (table, associations) - See Defining models

As a general rule, Model manages the state of a specific instance, and Collection of the entire collection of records. So for example, model.markAsDeleted() changes the local state of called record, but then delegates to its collection to notify collection observers and actually remove from the database

Query is a helper object that gives us a nice API to perform queries (query.observe(), query.fetchCount()):

• created via collection.query()
• encapsulates a QueryDescription structure which actually describes the query conditions
• fetch/observe methods actually delegate to Collection to perform database operations
• caches Observables created by observe/observeCount methods so they can be reused and shared

Helper functions

Watermelon's objects and classes are meant to be as minimal as possible — only manage their own state and be an API for your app. Most logic should be stateless, and implemented as pure functions:

QueryDescription is a structure (object) describing the query, built using Q.* helper functions

encodeMatcher(), simpleObserver(), reloadingObserver(), fieldObserver() implement query observation logic (See Observation.)

Model decorators transform simple class properties into Watermelon-aware record fields.

Much of Adapters' logic is implemented as pure functions too. See Adapters.

Database adapters

The idea for the Watermelon architecture is to be database-agnostic. Watermelon is a cross-platform high-level layer for dealing with data, but can be plugged in to any underlying database, depending on platform needs.

Think of it this way:

• Collection/Model/Query is the reactive layer
• DatabaseAdapter is the imperative layer

The adapter merely performs simple CRUD (create/read/update/delete) operations.

DatabaseAdapter is a Flow interface. Watermelon comes with two concrete implementations:

React Native

SQLiteAdapter is an adapter for React Native, based on SQLite:

• Queries are converted to SQL on app thread using adapters/sqlite/encodeQuery
• Communication happens over NativeModules with a native-side bridge
• Native database handling happens on a separate thread
• DatabaseBridge is the React Native bridge stub
• DatabaseDriver implements Watermelon-specific logic (caching, etc.)
• Database is a simple SQLite abstraction layer (over FMDB on iOS and built-in sqlite.SQLiteDatabase on Android)

Web

LokiJSAdapter is an adapter for the web, based around LokiJS:

• Why LokiJS? WebSQL would be a perfect fit for Watermelon, but sadly is a dead API, so we must use IndexedDB, but it's too low-level. LokiJS implements a fast querying API on top of IndexedDB.
• LokiJSAdapter delegates everything to a separate thread over WorkerBridge
• WorkerBridge spins up a worker thread running LokiWorker
• LokiWorker maintains a queue of operations and executes them on LokiExecutor
• LokiExecutor actually implements the Adapter operations
• encodeQuery translates QueryDescription objects to Loki query objects
• executeQuery implements join queries (Q.on), which Loki does not support

Writing your own adapter

If you want to write a new adapter, please contact @radex for more information.

⚠️ TODO: This section needs more concrete tips

Sync implementation details

If you're looking for a guide to implement Watermelon Sync in your app, see Synchronization.

If you want to contribute to Watermelon Sync, or implement your own synchronization engine from scratch, read this.

Implementing your own sync from scratch

For basic details about how changes tracking works, see: 📺 Digging deeper into WatermelonDB

Why you might want to implement a custom sync engine? If you have an existing remote server architecture that's difficult to adapt to Watermelon sync protocol, or you specifically want a different architecture (e.g. single HTTP request -- server resolves conflicts). Be warned, however, that implementing sync that works reliably is a hard problem, so we recommend sticking to Watermelon Sync and tweaking it as needed.

The rest of this document contains details about how Watermelon Sync works - you can use that as a blueprint for your own work.

If possible, please use sync implementation helpers from sync/*.js to keep your custom sync implementation have as much commonality as possible with the standard implementation. This is good both for you and for the rest of WatermelonDB community, as we get to share improvements and bug fixes. If the helpers are almost what you need, but not quite, please send pull requests with improvements!

Watermelon Sync -- Details

General design

• master/replica - server is the source of truth, client has a full copy and syncs back to server (no peer-to-peer syncs)
• two phase sync: first pull remote changes to local app, then push local changes to server
• client resolves conflicts
• content-based, not time-based conflict resolution
• conflicts are resolved using per-column client-wins strategy: in conflict, server version is taken except for any column that was changed locally since last sync.
• local app tracks its changes using a _status (synced/created/updated/deleted) field and _changes field (which specifies columns changed since last sync)
• server only tracks timestamps (or version numbers) of every record, not specific changes
• sync is performed for the entire database at once, not per-collection
• eventual consistency (client and server are consistent at the moment of successful pull if no local changes need to be pushed)
• non-blocking: local database writes (but not reads) are only momentarily locked when writing data but user can safely make new changes throughout the process

Sync procedure

1. Pull phase

• get last pulled at timestamp locally (null if first sync)
• call push changes function, passing lastPulledAt
  • server responds with all changes (create/update/delete) that occured since lastPulledAt
  • server serves us with its current timestamp
• IN ACTION (lock local writes):
  • ensure no concurrent syncs
  • apply remote changes locally
    • insert new records
      • if already exists (error), update
      • if locally marked as deleted (error), un-delete and update
    • update records
      • if synced, just replace contents with server version
      • if locally updated, we have a conflict!
        • take remote version, apply local fields that have been changed locally since last sync (per-column client wins strategy)
        • record stays marked as updated, because local changes still need to be pushed
      • if locally marked as deleted, ignore (deletion will be pushed later)
      • if doesn't exist locally (error), create
    • destroy records
      • if alredy deleted, ignore
      • if locally changed, destroy anyway
      • ignore children (server ought to schedule children to be destroyed)
  • if successful, save server's timestamp as new lastPulledAt

2. Push phase

• Fetch local changes
  • Find all locally changed records (created/updated record + deleted IDs) for all collections
  • Strip _status, _changed
• Call push changes function, passing local changes object, and the new lastPulledAt timestamp
  • Server applies local changes to database, and sends OK
  • If one of the pushed records has changed on the server since lastPulledAt, push is aborted, all changes reverted, and server responds with an error
• IN ACTION (lock local writes):
  • markLocalChangesAsSynced:
    • take local changes fetched in previous step, and:
    • permanently destroy records marked as deleted
    • mark created/updated records as synced and reset their _changed field
    • note: do not mark record as synced if it changed locally since fetch local changes step (user could have made new changes that need syncing)

Notes

• This procedure is designed such that if sync fails at any moment, and even leaves local app in inconsistent (not fully synced) state, we should still achieve consistency with the next sync:
  • applyRemoteChanges is designed such that if all changes are applied, but lastPulledAt doesn't get saved — so during next pull server will serve us the same changes, second applyRemoteChanges will arrive at the same result
  • local changes before "fetch local changes" step don't matter at all - user can do anything
  • local changes between "fetch local changes" and "mark local changes as synced" will be ignored (won't be marked as synced) - will be pushed during next sync
  • if changes don't get marked as synced, and are pushed again, server should apply them the same way
  • remote changes between pull and push phase will be locally ignored (will be pulled next sync) unless there's a per-record conflict (then push fails, but next sync resolves both pull and push)

Migration Syncs

Schema versioning and migrations complicate sync, because a client might not be able to sync some tables and columns, but after upgrade to the newest version, it should be able to get consistent sync. To be able to do that, we need to know what's the schema version at which the last sync occured. Unfortunately, Watermelon Sync didn't track that from the first version, so backwards-compat is required.

synchronize({ migrationsEnabledAtVersion: XXX })

. . . .

LPA = last pulled at
MEA = migrationsEnabledAtVersion, schema version at which future migration support was introduced
LS = last synced schema version (may be null due to backwards compat)
CV = current schema version

LPA     MEA     LS      CV      migration   set LS=CV?   comment

null    X       X       10      null        YES          first sync. regardless of whether the app
                                                         is migration sync aware, we can note LS=CV
                                                         to fetch all migrations once available

100     null    X       X       null        NO           indicates app is not migration sync aware so
                                                         we're not setting LS to allow future migration sync

100     X       10      10      null        NO           up to date, no migration
100     9       9       10      {9-10}      YES          correct migration sync
100     9       null    10      {9-10}      YES          fallback migration. might not contain all
                                                         necessary migrations, since we can't know for sure
                                                         that user logged in at then-current-version==MEA

100     9       11      10      ERROR       NO           LS > CV indicates programmer error
100     11      X       10      ERROR       NO           MEA > CV indicates programmer error

Reference

This design has been informed by:

• 10 years of experience building synchronization at Nozbe
• Kinto & Kinto.js
  • https://github.com/Kinto/kinto.js/blob/master/src/collection.js
  • https://kintojs.readthedocs.io/en/latest/api/#fetching-and-publishing-changes
• Histo - https://github.com/mirkokiefer/syncing-thesis

Dig deeper into WatermelonDB

Details about how Watermelon works, how to hack and contribute

📺 Digging deeper into WatermelonDB — more architectural info about caching, observation, and sync

WatermelonDB Roadmap

From today to 1.0

WatermelonDB is currently in active development at Nozbe for use in advanced projects. It's mostly feature-complete. However, there are a few features left before we can call it 1.0.

v0.xxx

• Full transactionality (atomicity) support ???
• Field sanitizers
• Optimized tree deleting
• API improvements

v1.0

Everything above plus having at least one non-trivial app using WatermelonDB in production to verify its concepts

Beyond 1.0

• Replace withObservables HOC and Prefetching with a solution based on React 17 Suspense feature
• Query templates

Contributing guidelines

Before you send a pull request

1. Did you add or changed some functionality?

   Add (or modify) tests!

2. Check if the automated tests pass

   yarn ci:check
   

3. Format the files you changed

   yarn prettier
   

4. Mark your changes in CHANGELOG

   Put a one-line description of your change under Added/Changed section. See Keep a Changelog.

Running Watermelon in development

Download source and dependencies

git clone https://github.com/Nozbe/WatermelonDB.git
cd WatermelonDB
yarn

Developing Watermelon alongside your app

To work on Watermelon code in the sandbox of your app:

yarn dev

This will create a dev/ folder in Watermelon and observe changes to source files (only JavaScript files) and recompile them as needed.

Then in your app:

cd node_modules/@nozbe
rm -fr watermelondb
ln -s path-to-watermelondb/dev watermelondb

This will work in Webpack but not in Metro (React Native). Metro doesn't follow symlinks. Instead, you can compile WatermelonDB directly to your project:

DEV_PATH="/path/to/your/app/node_modules/@nozbe/watermelondb" yarn dev

Running tests

This runs Jest, ESLint and Flow:

yarn ci:check

You can also run them separately:

yarn test
yarn eslint
yarn flow

Editing files

We recommend VS Code with ESLint, Flow, and Prettier (with prettier-eslint enabled) plugins for best development experience. (To see lint/type issues inline + have automatic reformatting of code)

Editing native code

In native/ios and native/android you'll find the native bridge code for React Native.

It's recommended to use the latest stable version of Xcode / Android Studio to work on that code.

Integration tests

If you change native bridge code or adapter/sqlite code, it's recommended to run integration tests that run the entire Watermelon code with SQLite and React Native in the loop:

yarn test:ios
yarn test:android

Running tests manualy

• For iOS open the native/iosTest/WatermelonTester.xcworkspace project and hit Cmd+U.
• For Android open native/androidTest in AndroidStudio navigate to app/src/androidTest/java/com.nozbe.watermelonTest/BridgeTest and click green arrow near class BridgeTest

Native linting

Make sure the native code you're editing conforms to Watermelon standards:

yarn swiftlint
yarn ktlint

Native code troubleshooting

1. If test:ios fails in terminal:

• Run tests in Xcode first before running from terminal
• Make sure you have the right version of Xcode CLI tools set in Preferences -> Locations

1. Make sure you're on the most recent stable version of Xcode / Android Studio
2. Remove native caches:

• Xcode: ~/Library/Developer/Xcode/DerivedData:
• Android: .gradle and build folders in native/android and native/androidTest
• node_modules (because of React Native precompiled third party libraries)

Changelog

All notable changes to this project will be documented in this file.

Unreleased

0.17 - 2020-06-22

New features

• [Sync] Introducing Migration Syncs - this allows fully consistent synchronization when migrating between schema versions. Previously, there was no mechanism to incrementally fetch all remote changes in new tables and columns after a migration - so local copy was likely inconsistent, requiring a re-login. After adopting migration syncs, Watermelon Sync will request from backend all missing information. See Sync docs for more details.

• [iOS] Introducing a new native SQLite database integration, rewritten from scratch in C++, based on React Native's JSI (JavaScript Interface). It is to be considered experimental, however we intend to make it the default (and eventually, the only) implementation. In a later release, Android version will be introduced.

   The new adapter is up to 3x faster than the previously fastest `synchronous: true` option,
   however this speedup is only achieved with some unpublished React Native patches.
  
   To try out JSI, add `experimentalUseJSI: true` to `SQLiteAdapter` constructor.
  

• [Query] Added Q.experimentalSortBy(sortColumn, sortOrder), Q.experimentalTake(count), Q.experimentalSkip(count) methods - @Kenneth-KT

• Database.batch() can now be called with a single array of models

• [DX] Database.get(tableName) is now a shortcut for Database.collections.get(tableName)

• [DX] Query is now thenable - you can now use await query and await query.count instead of await query.fetch() and await query.fetchCount()

• [DX] Relation is now thenable - you can now use await relation instead of await relation.fetch()

• [DX] Exposed collection.db and model.db as shortcuts to get to their Database object

Changes

• [Hardening] Column and table names starting with __, Object property names (e.g. constructor), and some reserved keywords are now forbidden
• [DX] [Hardening] QueryDescription builder methods do tighter type checks, catching more bugs, and preventing users from unwisely passing unsanitized user data into Query builder methods
• [DX] [Hardening] Adapters check early if table names are valid
• [DX] Collection.find reports an error more quickly if an obviously invalid ID is passed
• [DX] Intializing Database with invalid model classes will now show a helpful error
• [DX] DatabaseProvider shows a more helpful error if used improperly
• [Sync] Sync no longer fails if pullChanges returns collections that don't exist on the frontend - shows a warning instead. This is to make building backwards-compatible backends less error-prone
• [Sync] [Docs] Sync documentation has been rewritten, and is now closer in detail to a formal specification
• [Hardening] database.collections.get() better validates passed value
• [Hardening] Prevents unsafe strings from being passed as column name/table name arguments in QueryDescription

Fixes

• [Sync] Fixed RangeError: Maximum call stack size exceeded when syncing large amounts of data - @leninlin
• [iOS] Fixed a bug that could cause a database operation to fail with an (6) SQLITE_LOCKED error
• [iOS] Fixed 'jsi/jsi.h' file not found when building at the consumer level. Added path $(SRCROOT)/../../../../../ios/Pods/Headers/Public/React-jsi to Header Search Paths (issue #691) - @victorbutler
• [Native] SQLite keywords used as table or column names no longer crash
• Fixed potential issues when subscribing to database, collection, model, queries passing a subscriber function with the same identity more than once

Internal

• Fixed broken adapter tests

0.15.1, 0.16.1-fix, 0.16.2 - 2020-06-03

This is a security patch for a vulnerability that could cause maliciously crafted record IDs to cause all or some of user's data to be deleted. More information available via GitHub security advisory

0.16.1 - 2020-05-18

Changes

• Database.unsafeResetDatabase() is now less unsafe — more application bugs are being caught

Fixes

• [iOS] Fix build in apps using Flipper
• [Typescript] Added type definition for setGenerator.
• [Typescript] Fixed types of decorators.
• [Typescript] Add Tests to test Types.
• Fixed typo in learn-to-use docs.
• [Typescript] Fixed types of changes.

Internal

• [SQLite] Infrastruture for a future JSI adapter has been added

0.16 - 2020-03-06

⚠️ Breaking

• experimentalUseIncrementalIndexedDB has been renamed to useIncrementalIndexedDB

Low breakage risk

• [adapters] Adapter API has changed from returning Promise to taking callbacks as the last argument. This won't affect you unless you call on adapter methods directly. database.adapter returns a new DatabaseAdapterCompat which has the same shape as old adapter API. You can use database.adapter.underlyingAdapter to get back SQLiteAdapter / LokiJSAdapter
• [Collection] Collection.fetchQuery and Collection.fetchCount are removed. Please use Query.fetch() and Query.fetchCount().

New features

• [SQLiteAdapter] [iOS] Add new synchronous option to adapter: new SQLiteAdapter({ ..., synchronous: true }). When enabled, database operations will block JavaScript thread. Adapter actions will resolve in the next microtask, which simplifies building flicker-free interfaces. Adapter will fall back to async operation when synchronous adapter is not available (e.g. when doing remote debugging)
• [LokiJS] Added new onQuotaExceededError?: (error: Error) => void option to LokiJSAdapter constructor. This is called when underlying IndexedDB encountered a quota exceeded error (ran out of allotted disk space for app) This means that app can't save more data or that it will fall back to using in-memory database only Note that this only works when useWebWorker: false

Changes

• [Performance] Watermelon internals have been rewritten not to rely on Promises and allow some fetch/observe calls to resolve synchronously. Do not rely on this -- external API is still based on Rx and Promises and may resolve either asynchronously or synchronously depending on capabilities. This is meant as a internal performance optimization only for the time being.
• [LokiJS] [Performance] Improved worker queue implementation for performance
• [observation] Refactored observer implementations for performance

Fixes

• Fixed a possible cause for "Record ID xxx#yyy was sent over the bridge, but it's not cached" error
• [LokiJS] Fixed an issue preventing database from saving when using experimentalUseIncrementalIndexedDB
• Fixed a potential issue when using database.unsafeResetDatabase()
• [iOS] Fixed issue with clearing database under experimental synchronous mode

New features (Experimental)

• [Model] Added experimental model.experimentalSubscribe((isDeleted) => { ... }) method as a vanilla JS alternative to Rx based model.observe(). Unlike the latter, it does not notify the subscriber immediately upon subscription.
• [Collection] Added internal collection.experimentalSubscribe((changeSet) => { ... }) method as a vanilla JS alternative to Rx based collection.changes (you probably shouldn't be using this API anyway)
• [Database] Added experimental database.experimentalSubscribe(['table1', 'table2'], () => { ... }) method as a vanilla JS alternative to Rx-based database.withChangesForTables(). Unlike the latter, experimentalSubscribe notifies the subscriber only once after a batch that makes a change in multiple collections subscribed to. It also doesn't notify the subscriber immediately upon subscription, and doesn't send details about the changes, only a signal.
• Added experimentalDisableObserveCountThrottling() to @nozbe/watermelondb/observation/observeCount that globally disables count observation throttling. We think that throttling on WatermelonDB level is not a good feature and will be removed in a future release - and will be better implemented on app level if necessary
• [Query] Added experimental query.experimentalSubscribe(records => { ... }), query.experimentalSubscribeWithColumns(['col1', 'col2'], records => { ... }), and query.experimentalSubscribeToCount(count => { ... }) methods

0.15 - 2019-11-08

Highlights

This is a massive new update to WatermelonDB! 🍉

• Up to 23x faster sync. You heard that right. We've made big improvements to performance. In our tests, with a massive sync (first login, 45MB of data / 65K records) we got a speed up of:

  • 5.7s -> 1.2s on web (5x)
  • 142s -> 6s on iOS (23x)

  Expect more improvements in the coming releases!

• Improved LokiJS adapter. Option to disable web workers, important Safari 13 fix, better performance, and now works in Private Modes. We recommend adding useWebWorker: false, experimentalUseIncrementalIndexedDB: true options to the LokiJSAdapter constructor to take advantage of the improvements, but please read further changelog to understand the implications of this.

• Raw SQL queries now available on iOS and Android thanks to the community

• Improved TypeScript support — thanks to the community

⚠️ Breaking

• Deprecated bool schema column type is removed -- please change to boolean
• Experimental experimentalSetOnlyMarkAsChangedIfDiffers(false) API is now removed

New featuers

• [Collection] Add Collection.unsafeFetchRecordsWithSQL() method. You can use it to fetch record using raw SQL queries on iOS and Android. Please be careful to avoid SQL injection and other pitfalls of raw queries

• [LokiJS] Introduces new new LokiJSAdapter({ ..., experimentalUseIncrementalIndexedDB: true }) option. When enabled, database will be saved to browser's IndexedDB using a new adapter that only saves the changed records, instead of the entire database.

  This works around a serious bug in Safari 13 (https://bugs.webkit.org/show_bug.cgi?id=202137) that causes large databases to quickly balloon to gigabytes of temporary trash

  This also improves performance of incremental saves, although initial page load or very, very large saves might be slightly slower.

  This is intended to become the new default option, but it's not backwards compatible (if enabled, old database will be lost). You're welcome to contribute an automatic migration code.

  Note that this option is still experimental, and might change in breaking ways at any time.

• [LokiJS] Introduces new new LokiJSAdapter({ ..., useWebWorker: false }) option. Before, web workers were always used with LokiJSAdapter. Although web workers may have some performance benefits, disabling them may lead to lower memory consumption, lower latency, and easier debugging. YMMV.

• [LokiJS] Added onIndexedDBVersionChange option to LokiJSAdapter. This is a callback that's called when internal IDB version changed (most likely the database was deleted in another browser tab). Pass a callback to force log out in this copy of the app as well. Note that this only works when using incrementalIDB and not using web workers

• [Model] Add Model._dangerouslySetRawWithoutMarkingColumnChange() method. You probably shouldn't use it, but if you know what you're doing and want to live-update records from server without marking record as updated, this is useful

• [Collection] Add Collection.prepareCreateFromDirtyRaw()

• @json decorator sanitizer functions take an optional second argument, with a reference to the model

Fixes

• Pinned required rambdax version to 2.15.0 to avoid console logging bug. In a future release we will switch to our own fork of rambdax to avoid future breakages like this.

Improvements

• [Performance] Make large batches a lot faster (1.3s shaved off on a 65K insert sample)
• [Performance] [iOS] Make large batch inserts an order of magnitude faster
• [Performance] [iOS] Make encoding very large queries (with thousands of parameters) 20x faster
• [Performance] [LokiJS] Make batch inserts faster (1.5s shaved off on a 65K insert sample)
• [Performance] [LokiJS] Various performance improvements
• [Performance] [Sync] Make Sync faster
• [Performance] Make observation faster
• [Performance] [Android] Make batches faster
• Fix app glitches and performance issues caused by race conditions in Query.observeWithColumns()
• [LokiJS] Persistence adapter will now be automatically selected based on availability. By default, IndexedDB is used. But now, if unavailable (e.g. in private mode), ephemeral memory adapter will be used.
• Disabled console logs regarding new observations (it never actually counted all observations) and time to query/count/batch (the measures were wildly inaccurate because of asynchronicity - actual times are much lower)
• [withObservables] Improved performance and debuggability (update withObservables package separately)
• Improved debuggability of Watermelon -- shortened Rx stacks and added function names to aid in understanding call stacks and profiles
• [adapters] The adapters interface has changed. query() and count() methods now receive a SerializedQuery, and batch() now takes TableName<any> and RawRecord or RecordId instead of Model.
• [Typescript] Typing improvements
  • Added 3 missing properties collections, database and asModel in Model type definition.
  • Removed optional flag on actionsEnabled in the Database constructor options since its mandatory since 0.13.0.
  • fixed several further typing issues in Model, Relation and lazy decorator
• Changed how async functions are transpiled in the library. This could break on really old Android phones but shouldn't matter if you use latest version of React Native. Please report an issue if you see a problem.
• Avoid database prop drilling in the web demo

0.14.1 - 2019-08-31

Hotfix for rambdax crash

• [Schema] Handle invalid table schema argument in appSchema
• [withObservables] Added TypeScript support (changelog)
• [Electron] avoid Uncaught ReferenceError: global is not defined in electron runtime (#453)
• [rambdax] Replaces contains with includes due to contains deprecation https://github.com/selfrefactor/rambda/commit/1dc1368f81e9f398664c9d95c2efbc48b5cdff9b#diff-04c6e90faac2675aa89e2176d2eec7d8R2209

0.14.0 - 2019-08-02

New features

• [Query] Added support for notLike queries 🎉
• [Actions] You can now batch delete record with all descendants using experimental functions experimentalMarkAsDeleted or experimentalDestroyPermanently

0.13.0 - 2019-07-18

⚠️ Breaking

• [Database] It is now mandatory to pass actionsEnabled: option to Database constructor. It is recommended that you enable this option:

  const database = new Database({
    adapter: ...,
    modelClasses: [...],
    actionsEnabled: true
  })
  

  See docs/Actions.md for more details about Actions. You can also pass false to maintain backward compatibility, but this option will be removed in a later version

• [Adapters] migrationsExperimental prop of SQLiteAdapter and LokiJSAdapter has been renamed to migrations.

New features

• [Actions] You can now batch deletes by using prepareMarkAsDeleted or prepareDestroyPermanently
• [Sync] Performance: synchronize() no longer calls your pushChanges() function if there are no local changes to push. This is meant to save unnecessary network bandwidth. ⚠️ Note that this could be a breaking change if you rely on it always being called
• [Sync] When setting new values to fields on a record, the field (and record) will no longer be marked as changed if the field's value is the same. This is meant to improve performance and avoid unnecessary code in the app. ⚠️ Note that this could be a breaking change if you rely on the old behavior. For now you can import experimentalSetOnlyMarkAsChangedIfDiffers from @nozbe/watermelondb/Model/index and call if with (false) to bring the old behavior back, but this will be removed in the later version -- create a new issue explaining why you need this
• [Sync] Small perf improvements

Improvements

• [Typescript] Improved types for SQLite and LokiJS adapters, migrations, models, the database and the logger.

0.12.3 - 2019-05-06

Changes

• [Database] You can now update the random id schema by importing import { setGenerator } from '@nozbe/watermelondb/utils/common/randomId' and then calling setGenerator(newGenenerator). This allows WatermelonDB to create specific IDs for example if your backend uses UUIDs.
• [Typescript] Type improvements to SQLiteAdapter and Database
• [Tests] remove cleanup for react-hooks-testing-library@0.5.0 compatibility

0.12.2 - 2019-04-19

Fixes

• [TypeScript] 'Cannot use 'in' operator to search for 'initializer'; decorator fix

Changes

• [Database] You can now pass falsy values to Database.batch(...) (false, null, undefined). This is useful in keeping code clean when doing operations conditionally. (Also works with model.batch(...))
• [Decorators]. You can now use @action on methods of any object that has a database: Database property, and @field @children @date @relation @immutableRelation @json @text @nochange decorators on any object with a asModel: Model property.
• [Sync] Adds a temporary/experimental _unsafeBatchPerCollection: true flag to synchronize(). This causes server changes to be committed to database in multiple batches, and not one. This is NOT preferred for reliability and performance reasons, but it works around a memory issue that might cause your app to crash on very large syncs (>20,000 records). Use this only if necessary. Note that this option might be removed at any time if a better solution is found.

0.12.1 - 2019-04-01

⚠️ Hotfix

• [iOS] Fix runtime crash when built with Xcode 10.2 (Swift 5 runtime).

  ⚠️ Note: You need to upgrade to React Native 0.59.3 for this to work. If you can't upgrade React Native yet, either stick to Xcode 10.1 or manually apply this patch: https://github.com/Nozbe/WatermelonDB/pull/302/commits/aa4e08ad0fa55f434da2a94407c51fc5ff18e506

Changes

• [Sync] Adds basic sync logging capability to Sync. Pass an empty object to synchronize() to populate it with diagnostic information:

  const log = {}
  await synchronize({ database, log, ...})
  console.log(log.startedAt)
  

  See Sync documentation for more details.

0.12.0 - 2019-03-18

Added

• [Hooks] new useDatabase hook for consuming the Database Context:

  import { useDatabase } from '@nozbe/watermelondb/hooks';
  const Component = () => {
     const database = useDatabase();
  }
  

• [TypeScript] added .d.ts files. Please note: TypeScript definitions are currently incomplete and should be used as a guide only. PRs for improvements would be greatly appreciated!

Performance

• Improved UI performance by consolidating multiple observation emissions into a single per-collection batch emission when doing batch changes

0.11.0 - 2019-03-12

Breaking

• ⚠️ Potentially BREAKING fix: a @date field now returns a Jan 1, 1970 date instead of null if the field's raw value is 0. This is considered a bug fix, since it's unexpected to receive a null from a getter of a field whose column schema doesn't say isOptional: true. However, if you relied on this behavior, this might be a breaking change.
• ⚠️ BREAKING: Database.unsafeResetDatabase() now requires that you run it inside an Action

Bug fixes

• [Sync] Fixed an issue where synchronization would continue running despite unsafeResetDatabase being called
• [Android] fix compile error for kotlin 1.3+

Other changes

• Actions are now aborted when unsafeResetDatabase() is called, making reseting database a little bit safer
• Updated demo dependencies
• LokiJS is now a dependency of WatermelonDB (although it's only required for use on the web)
• [Android] removed unused test class
• [Android] updated ktlint to 0.30.0

0.10.1 - 2019-02-12

Changes

• [Android] Changed compile to implementation in Library Gradle file
  • ⚠️ might break build if you are using Android Gradle Plugin <3.X
• Updated peerDependency react-native to 0.57.0
• [Sync] Added hasUnsyncedChanges() helper method
• [Sync] Improved documentation for backends that can't distinguish between created and updated records
• [Sync] Improved diagnostics / protection against edge cases
• [iOS] Add missing header search path to support ejected expo project.
• [Android] Fix crash on android < 5.0
• [iOS] SQLiteAdapter's dbName path now allows you to pass an absolute path to a file, instead of a name
• [Web] Add adaptive layout for demo example with smooth scrolling for iOS

0.10.0 - 2019-01-18

Breaking

• BREAKING: Table column last_modified is no longer automatically added to all database tables. If you don't use this column (e.g. in your custom sync code), you don't have to do anything. If you do, manually add this column to all table definitions in your Schema:

  { name: 'last_modified', type: 'number', isOptional: true }
  

  Don't bump schema version or write a migration for this.

New

• Actions API.

  This was actually released in 0.8.0 but is now documented in CRUD.md and Actions.md. With Actions enabled, all create/update/delete/batch calls must be wrapped in an Action.

  To use Actions, call await database.action(async () => { /* perform writes here */ }, and in Model instance methods, you can just decorate the whole method with @action.

  This is necessary for Watermelon Sync, and also to enable greater safety and consistency.

  To enable actions, add actionsEnabled: true to new Database({ ... }). In a future release this will be enabled by default, and later, made mandatory.

  See documentation for more details.

• Watermelon Sync Adapter (Experimental)

  Added synchronize() function that allows you to easily add full synchronization capabilities to your Watermelon app. You only need to provide two fetch calls to your remote server that conforms to Watermelon synchronization protocol, and all the client-side processing (applying remote changes, resolving conflicts, finding local changes, and marking them as synced) is done by Watermelon.

  See documentation for more details.

• Support caching for non-global IDs at Native level

0.9.0 - 2018-11-23

New

• Added Q.like - you can now make queries similar to SQL LIKE

0.8.0 - 2018-11-16

New

• Added DatabaseProvider and withDatabase Higher-Order Component to reduce prop drilling
• Added experimental Actions API. This will be documented in a future release.

Fixes

• Fixes crash on older Android React Native targets without jsc-android installed

0.7.0 - 2018-10-31

Deprecations

• [Schema] Column type 'bool' is deprecated — change to 'boolean'

New

• Added support for Schema Migrations. See documentation for more details.
• Added fundaments for integration of Danger with Jest

Changes

• Fixed "dependency cycle" warning
• [SQLite] Fixed rare cases where database could be left in an unusable state (added missing transaction)
• [Flow] Fixes oneOf() typing and some other variance errors
• [React Native] App should launch a little faster, because schema is only compiled on demand now
• Fixed typos in README.md
• Updated Flow to 0.85

0.6.2 - 2018-10-04

Deprecations

• The @nozbe/watermelondb/babel/cjs / @nozbe/watermelondb/babel/esm Babel plugin that ships with Watermelon is deprecated and no longer necessary. Delete it from your Babel config as it will be removed in a future update

Refactoring

• Removed dependency on async (Web Worker should be ~30KB smaller)
• Refactored Collection and simpleObserver for getting changes in an array and also adds CollectionChangeTypes for differentiation between different changes
• Updated dependencies
• Simplified build system by using relative imports
• Simplified build package by outputting CJS-only files

0.6.1 - 2018-09-20

Added

• Added iOS and Android integration tests and lint checks to TravisCI

Changed

• Changed Flow setup for apps using Watermelon - see docs/Advanced/Flow.md
• Improved documentation, and demo code
• Updated dependencies

Fixed

• Add quotes to all names in sql queries to allow keywords as table or column names
• Fixed running model tests in apps with Watermelon in the loop
• Fixed Flow when using Watermelon in apps

0.6.0 - 2018-09-05

Initial release of WatermelonDB