Quick StartAPI Reference

8. Working with the DataStore

The DataStore component is mostly used in Single-Page Applications

Suggest Edits

🚧

This tutorial needs review

Getting started

This page assumes you've read part 7 of the tutorial, Relations.

📘

Info

The DataStore or SimpleStore classes are meant for use in a frontend Single Page Application.

Of all the JSData components, the DataStore and SimpleStore components are at the top of the food chain. They are opinionated, in-memory caches that manage your Adapters, Mappers, Collections, and Records. DataStore and SimpleStore combine all of the other JSData classes to implement the "in-memory data store" functionality.

👍

Tip

Instead of using DataStore or SimpleStore classes, you can implement your own "store" using some combination of the Container, Mapper, Schema, Collection, and Record classes.

DataStore vs SimpleStore

SimpleStore

The SimpleStore class extends the Container class and adds caching. For each Mapper that you define, SimpleStore creates a corresponding Collection instance and uses it to cache the results of SimpleStore#find() and SimpleStore#findAll() queries. Calls to SimpleStore's other CRUD methods also update the Records cached in the Collections.

SimpleStore proxies all Collection methods, requiring that you specify which Mapper to target when calling the method.

1760

DataStore

The DataStore class extends the SimpleStore class, swapping the Collection class for the LinkedCollection class and overriding a few SimpleStore methods to add "relation linking" functionality.

1760

📘

Info

Everything that is true for SimpleStore in this document is also true for DataStore

Async vs Sync

All SimpleStore asynchronous methods are inherited from the Container, which proxies the methods of its Mappers. Likewise, SimpleStore proxies the methods of its Collections.

The asynchronous methods are used to perform CRUD operations and are executed using the configured adapter, hence why they are asynchronous. The asynchronous CRUD methods have corresponding synchronous methods that target the in-memory Records only.

AsyncSync
create, createMany, update, updateMany, updateAlladd
findget
findAllfilter, getAll
destroyremove
destroyAllremoveAll

👍

Tip

Use the asynchronous methods to perform CRUD operations that persist changes via your configured adapter. Use the synchronous methods to read and manipulate in-memory data only.

Loading data into the store

There are several ways to load Records into the store:

  • find() loads a single Record asynchronously via your configured adapter.
  • findAll() loads a collection of Records asynchronously via your configured adapter.
  • add() synchronously inserts one or more Records directly into the store
const id = 1234;

// "find" returns a promise
store.find('post', id)
  // Assuming the adapter is able to retrieve a single
  // "post" record, then the record is inserted into
  // the store and the promise resolves with the record:
  .then((post) => {
    // {
    //   id: 1234,
    //   title: 'JSData'
    // }
    console.log(post);

    // The post is now in the store:
    console.log(store.get('post', id));
  });

const query = { post_id: 1234 };

// "findAll" returns a promise
store.findAll('comment', query)
  // Assuming the adapter is able to retrieve some
  // "comment" records, then the records are inserted into
  // the store and the promise resolves with the records:
  .then((comments) => {
    // [
    //   {
    //     id: 33,
    //     content: 'Sweet!',
    //     post_id: 1234
    //   },
    //   {
    //     id: 44,
    //     content: 'This is awesome!',
    //     post_id: 1234
    //   }
    // ]
    console.log(comments);
  
    // The comments are now in the store:
    console.log(store.filter('comment', { post_id: 1234 }));
    console.log(store.getAll('comment', 1234, { index: 'post_id' }));
  });

// "add" returns the inserted Record(s)
const post = store.add('post', { id: 1234, title: 'JSData' });

// The post is now in the store:
post === store.get('post', post.id); // true

const comments = store.add('comment', [
  { id: 33, content: 'Sweet!', post_id: 1234 },
  { id: 44, content: 'This is awesome!', post_id: 1234 }
]);

// The comments are now in the store:
assert.deepEqual(
  comments,
  store.filter('comment', { post_id: 1234 })
);
assert.deepEqual(
  comments,
  store.getAll('comment', 1234, { index: 'post_id' })
);

Relations

DataStore uses the LinkedCollection class and is capable of distributing embedded relations into the correct part of the store as Records are inserted.

// Must be using the DataStore class
store instanceof DataStore; // true

const id = 1234;

// Let's assume the adapter retrieves the following:
//
// {
//   id: 1234,
//   title: 'JSData',
//   comments: [
//     { id: 33, content: 'Sweet!', post_id: 1234 },
//     { id: 44, content: 'This is awesome!', post_id: 1234 }
//   ]
// }
store.find('post', id)
  .then((post) => {
    // When the post was inserted into the store, its
    // comments were inserted into their part of the store
    // as well:
    post === store.get('post', id); // true

    // The comments are now in the store:
    console.log(store.filter('comment', { post_id: 1234 }));
    console.log(store.getAll('comment', 1234, { index: 'post_id' }));
  });

// Must be using the DataStore class
store instanceof DataStore; // true

const query = { post_id: 1234 };
const options = { with: ['user'] };

// Note: If using the HTTP adapter, perhaps do this instead:
// const options = { params: { with: ['user'] } };
//
// See http://stackoverflow.com/a/41926444/1153216

// Let's assume the adapter retrieves the following:
//
// [
//   {
//     id: 33,
//     content: 'Sweet!',
//     post_id: 1234,
//     user_id: 55
//     user: {
//       id: 55,
//       name: 'Bob'
//     }
//   },
//   {
//     id: 44,
//     content: 'This is awesome!',
//     post_id: 1234,
//     user_id: 66
//     user: {
//       id: 66,
//       name: 'Alice'
//     }
//   }
// ]
store.findAll('comment', query, options)
  // Assuming the adapter is able to retrieve some
  // "comment" records, then the records are inserted into
  // the store and the promise resolves with the records:
  .then((comments) => {
    // When the comments were inserted into the store, their
    // users were inserted into their part of the store
    // as well:
    console.log(store.filter('comment', { post_id: 1234 }));
    console.log(store.getAll('comment', 1234, { index: 'post_id' }));
  
    comments[0].user === store.get('user', comments[0].user_id); // true
    comments[1].user === store.get('user', comments[1].user_id); // true
  });

// Must be using the DataStore class
store instanceof DataStore; // true

// "add" returns the inserted Record(s)
const post = store.add('post', {
  id: 1234,
  title: 'JSData',
  comments: [
    { id: 33, content: 'Sweet!', post_id: 1234 },
    { id: 44, content: 'This is awesome!', post_id: 1234 }
  ]
});

// The post is now in the store:
post === store.get('post', post.id); // true

// The comments are now in the store:
assert.deepEqual(
  post.comments,
  store.filter('comment', { post_id: 1234 })
);
assert.deepEqual(
  post.comments,
  store.getAll('comment', 1234, { index: 'post_id' })
);

👍

Tip

If you've manually loaded data into your application, use add() to insert the data into the store and simultaneously convert it to Record instances.

Caching queries

Caching the results of find():

This is what happens when you call find(), using store.find('post', 1234) as the example:

  1. The store looks up any previous find() call to post 1234 that is still pending:
  • If there is no previous call to post 1234 that is still pending, then find() continues to step 2.
  • If there is a previous call to post 1234 that is still pending (the promise hasn't resolved yet):
    • If usePendingFind is true, then find() simply returns the pending promise.
    • If usePendingFind is false, then find() continues to step 2.
    • If usePendingFind is a function, then call it and pass it the arguments that were passed to find(). If it returns true, then find() returns the pending promise. If it returns false, then find() continues to step 2.
  1. find() calls the store's cachedFind() method, passing it the arguments that were passed to find(). The purpose of the cachedFind() method is to decide whether the find('post', 1234) call has already been cached. cachedFind() should either return the cached record or undefined.
  • If cachedFind() returns the cached record, then find() immediately returns a promise resolved with the cached record.
  • If cachedFind() returns a falsy value or a force: true option was passed to find():
    1. find() calls Container#find().
    2. The store keeps track of the pending promise returned by Container#find().
    3. When the pending promise resolves:
    4. addToCache() is called with the result. This method inserts the record into the store.
    5. cacheFind() is called with the result. This method marks this particular find() query (post 1234) as cached.
    6. Finally find() returns the promise, giving the caller access to the value returned by the call toaddToCache().

These steps represent the default, opinionated behavior, but there are some points of customization:

  • Override the cachedFind() method to change how the store decides whether a particular find() query has been cached.
  • Override the addToCache() method to change how the results received from Container#find() are inserted into the store.
  • Override the cacheFind() method to change how the find() query is marked as cached.
store.defineMapper('post', {
  // The default is true, unless you've changed that via
  // Container#mapperDefaults#usePendingFind
  usePendingFind: true
});

const promiseOne = store.find('post', 1234);
const promiseTwo = store.find('post', 1234);

promiseOne === promiseTwo; // true

const promiseThree = store.find('post', 1234, { usePendingFind: false });

promiseTwo === promiseThree; // false

store.defineMapper('comment', {
  usePendingFind: false
});

const promiseFour = store.find('comment', 33);
const promiseFive = store.find('comment', 33);

promiseFour === promiseFive; // false

const promiseSix = store.find('comment', 33, { usePendingFind: true });

promiseFive === promiseSix; // true

store.find('post', 1234)
  .then((post) => {
    // No new request will be made, the promise will
    // resolve immediately with the cached record:
    return store.find('post', 1234);
  })
  .then((post) => {
    // Ignore the cached record and force a new request:
    return store.find('post', 1234, { force: true });
  })
  // Container#find() was called again
  .then((post) => {
    // ...
  });

class PostMapper extends Mapper {
  cachedFind (mapperName, id, opts) {
    // You can put whatever kind of custom behavior
    // here that you want. cachedFind() should return
    // "undefined" if you don't want find() to use
    // the cached record. If you do want find() to
    // use the cached record, then cachedFind() should
    // return the cached record that find() should
    // use. The default behavior of cachedFind() is
    // the following:
    //
    // const cached = this._completedQueries[mapperName][id]
    // if (typeof cached === 'function') {
    //   return cached(mapperName, id, opts);
    // }
    // return cached;
    
    // For example, this will prevent find() from
    // ever using a cached record, forcing find()
    // to always make a new request:
    return undefined;
  }
}

store.defineMapper('post', {
  mapperClass: PostMapper
});

class PostMapper extends Mapper {
  addToCache (mapperName, data, opts) {
    // You can put whatever kind of custom behavior
    // here that you want. The default behavior of
    // addToCache() is the following:
    //
    // return this.getCollection(name).add(data, opts);
    
    return super.addToCache(mapperName, data, opts);
  }
}

store.defineMapper('post', {
  mapperClass: PostMapper
});

class PostMapper extends Mapper {
  cacheFind (mapperName, data, id, opts) {
    // You can put whatever kind of custom behavior
    // here that you want. The default behavior of
    // cacheFind() is the following:
    //
    // this._completedQueries[mapperName][id] = (mapperName, id, opts) => {
    //   return this.get(mapperName, id);
    // };
    
    return super.cacheFind(mapperName, data, id, opts);
  }
}

store.defineMapper('post', {
  mapperClass: PostMapper
});

Caching the results of findAll():

This is what happens when you call findAll(), using store.findAll('post', { status: 'draft' }) as the example:

  1. The store looks up any previous findAll() call to post { status: 'draft' } that is still pending:
  • If there is no previous call to post { status: 'draft' } that is still pending, then findAll() continues to step 2.
  • If there is a previous call to post { status: 'draft' } that is still pending (the promise hasn't resolved yet):
    • If usePendingFindAll is true, then findAll() simply returns the pending promise.
    • If usePendingFindAll is false, then findAll() continues to step 2.
    • If usePendingFindAll is a function, then call it and pass it the arguments that were passed to findAll(). If it returns true, then findAll() returns the pending promise. If it returns false, then findAll() continues to step 2.
  1. findAll() calls the store's cachedFindAll() method, passing it the arguments that were passed to findAll(). The purpose of the cachedFindAll() method is to decide whether the findAll('post', { status: 'draft' }) call has already been cached. cachedFindAll() should either return the cached records or undefined.
  • If cachedFindAll() returns the cached records, then findAll() immediately returns a promise resolved with the cached records.
  • If cachedFindAll() returns a falsy value or a force: true option was passed to findAll():
    1. findAll() calls Container#findAll().
    2. The store keeps track of the pending promise returned by Container#findAll().
    3. When the pending promise resolves:
    4. addToCache() is called with the result. This method inserts the records into the store.
    5. cacheFindAll() is called with the result. This method marks this particular findAll() query (post { status: 'draft' }) as cached.
    6. Finally findAll() returns the promise, giving the caller access to the value returned by the call toaddToCache().

These steps represent the default, opinionated behavior, but there are some points of customization:

  • Override the cachedFindAll() method to change how the store decides whether a particular findAll() query has been cached.
  • Override the addToCache() method to change how the results received from Container#findAll() are inserted into the store.
  • Override the cacheFindAll() method to change how the findAll() query is marked as cached.
store.defineMapper('post', {
  // The default is true, unless you've changed that via
  // Container#mapperDefaults#usePendingFindAll
  usePendingFindAll: true
});

const promiseOne = store.findAll('post', { status: 'draft' });
const promiseTwo = store.findAll('post', { status: 'draft' });

promiseOne === promiseTwo; // true

const promiseThree = store.findAll('post', { status: 'draft' }, { usePendingFindAll: false });

promiseTwo === promiseThree; // false

store.defineMapper('comment', {
  usePendingFindAll: false
});

const promiseFour = store.findAll('comment', { post_id: 1234 });
const promiseFive = store.findAll('comment', { post_id: 1234 });

promiseFour === promiseFive; // false

const promiseSix = store.findAll('comment', { post_id: 1234 }, { usePendingFindAll: true });

promiseFive === promiseSix; // true

store.findAll('post', { status: 'draft' })
  .then((posts) => {
    // No new request will be made, the promise will
    // resolve immediately with the cached records:
    return store.findAll('post', { status: 'draft' });
  })
  .then((posts) => {
    // Ignore the cached records and force a new request:
    return store.findAll('post', { status: 'draft' }, { force: true });
  })
  // Container#findAll() was called again
  .then((posts) => {
    // ...
  });

class PostMapper extends Mapper {
  cachedFindAll (mapperName, queryHash, opts) {
    // You can put whatever kind of custom behavior
    // here that you want. cachedFindAll() should return
    // "undefined" if you don't want findAll() to use
    // the cached records. If you do want findAll() to
    // use the cached records, then cachedFindAll() should
    // return the cached records that findAll() should
    // use. The default behavior of cachedFindAll() is
    // the following:
    //
    // const cached = this._completedQueries[mapperName][queryHash]
    // if (typeof cached === 'function') {
    //   return cached(mapperName, queryHash, opts);
    // }
    // return cached;
    
    // For example, this will prevent findAll() from
    // ever using cached records, forcing findAll()
    // to always make a new request:
    return undefined;
  }
}

store.defineMapper('post', {
  mapperClass: PostMapper
});

class PostMapper extends Mapper {
  addToCache (mapperName, data, opts) {
    // You can put whatever kind of custom behavior
    // here that you want. The default behavior of
    // addToCache() is the following:
    //
    // return this.getCollection(name).add(data, opts);
    
    return super.addToCache(mapperName, data, opts);
  }
}

store.defineMapper('post', {
  mapperClass: PostMapper
});

class PostMapper extends Mapper {
  cacheFindAll (mapperName, data, queryHash, opts) {
    // You can put whatever kind of custom behavior
    // here that you want. The default behavior of
    // cacheFindAll() is the following:
    //
    // this._completedQueries[mapperName][queryHash] = (mapperName, queryHash, opts) => {
    //   return this.filter(mapperName, utils.fromJson(queryHash));
    // };
    
    return super.cacheFindAll(mapperName, data, queryHash, opts);
  }
}

store.defineMapper('post', {
  mapperClass: PostMapper
});

Reading data from the store

A store holds Records using Collection instances. A Collection instance is basically a super fancy array that allows looking up Records several ways. Here are some examples:

// SimpleStore#get looks up a single record by id:
// http://api.js-data.io/js-data/latest/SimpleStore.html#get

store.get('post', 1234); // undefined, oops, it's not in the store yet

const insertedPost = store.add('post', { id: 1234, title: 'JSData' });

const post = store.get('post', 1234); // {...}

insertedPost === post // true! what?

// Looking up the post again gives you a reference
// to the same object, because the store is an
// Identity Map:
insertedPost === post // true!
insertedPost === store.get('post', 1234); // true!
post === store.get('post', 1234); // true!

// SimpleStore#filter looks up records that match a query:
// http://api.js-data.io/js-data/latest/SimpleStore.html#filter
store.filter('post'); // [], oops, no posts in the store yet

const insertedPosts = store.add('post', [
  { id: 1, title: 'JSData', status: 'published' },
  { id: 2, title: 'JSDataHttp', status: 'draft' },
  { id: 3, title: 'JSDataRethinkDB', status: 'draft' },
  { id: 4, title: 'JSDataFirebase', status: 'draft' },
  { id: 5, title: 'JSDataHttp', status: 'draft' },
  { id: 6, title: 'JSDataSql', status: 'draft' }
]);

store.filter('post'); // [{...}, {...}, {...}, {...}, {...}, {...}]
store.filter('post', { status: 'draft' }); // [{...}, {...}, {...}, {...}, {...}]
store.filter('post', { status: 'published' }); // [{...}]

// filter() also lets you sort, limit, and offset the results
store.filter('post', { status: 'draft', limit: 3 }); // [{...}, {...}, {...}]
store.filter('post', { status: 'draft', limit: 3, skip: 4 }); // [{...}, {...}]

// What will the result of this be?
const posts = store.filter('post', {
  where: {
    status: {
    	'==': 'draft'
    },
    id: {
      '>=': 4
    }
  },
  orderBy: [
    ['id', 'desc']
  ],
  limit: 1,
  skip: 1
});

posts.length; // 1
console.log(posts[0]); // { id: 5, title: 'JSDataHttp', status: 'draft' }

// SimpleStore#query is the builder-style version of
// filter() with some extra features:
// http://api.js-data.io/js-data/latest/SimpleStore.html#query
store
  .query('post')
  .run(); // [], oops, no posts in the store yet

const insertedPosts = store.add('post', [
  { id: 1, title: 'JSData', status: 'published' },
  { id: 2, title: 'JSDataHttp', status: 'draft' },
  { id: 3, title: 'JSDataRethinkDB', status: 'draft' },
  { id: 4, title: 'JSDataFirebase', status: 'draft' },
  { id: 5, title: 'JSDataHttp', status: 'draft' },
  { id: 6, title: 'JSDataSql', status: 'draft' }
]);

store
  .query('post')
  .run(); // [{...}, {...}, {...}, {...}, {...}, {...}]
store
  .query('post')
  .filter({ status: 'draft' })
  .run(); // [{...}, {...}, {...}, {...}, {...}]
store
  .query('post')
  .filter({ status: 'published' })
  .run(); // [{...}]

// filter() also lets you sort, limit, and offset the results
store
  .query('post')
  .filter({ status: 'draft' })
  .limit(3); // [{...}, {...}, {...}]
store
  .query('post')
  .filter({ status: 'draft' })
  .limit(3)
  .skip(4); // [{...}, {...}]

// What will the result of this be?
const posts = store
  .query('post')
  .filter({
    where: {
      status: {
      	'==': 'draft'
      },
      id: {
        '>=': 4
      }
    },
    orderBy: [
      ['id', 'desc']
    ]
  })
  .limit(1)
  .skip(1)
  .map((post) => post.title)
  .run();

posts.length; // 1
console.log(posts[0]); // "JSDataHttp"

store.defineMapper('post', {
  schema: {
    type: 'object',
    properties: {
      id: { type: 'string' },
      title: { type: 'string' },
      user_id: { type: 'number', indexed: true }
    }
  }
});

// SimpleStore#getAll looks up records by primary or secondary keys
// http://api.js-data.io/js-data/latest/SimpleStore.html#getAll
store.getAll('post'); // [], oops, no posts in the store yet

const insertedPosts = store.add('post', [
  { id: 1, title: 'JSData', user_id: 22 },
  { id: 2, title: 'JSDataHttp', user_id: 44 },
  { id: 3, title: 'JSDataRethinkDB', user_id: 44 },
  { id: 4, title: 'JSDataFirebase', user_id: 44 },
  { id: 5, title: 'JSDataHttp', user_id: 33 },
  { id: 6, title: 'JSDataSql', user_id: 33 }
]);

// By default it uses the primary key to lookup records
store.getAll('post'); // [{...}, {...}, {...}, {...}, {...}, {...}]
store.getAll('post', 1); // [{...}]
store.getAll('post', 4); // [{...}]

// You can specify a secondary key
store.getAll('post', 22, { index: 'user_id' }); // [{...}]
store.getAll('post', 44, { index: 'user_id' }); // [{...}, {...}, {...}]
store.getAll('post', 33, { index: 'user_id' }); // [{...}, {...}]

// SimpleStore#unsaved looks up records that do not have primary keys
// http://api.js-data.io/js-data/latest/SimpleStore.html#unsaved

// Insert a post that does have a primary key
store.add({ id: 1234, title: 'JSData', status: 'published' });

store.filter('post'); // [{...}]
store.unsaved('post'); // []

// Insert a post that doesn't have a primary key yet
store.add({ title: 'my new post', status: 'draft' });

// The unsaved post can't be referenced by primary key:
store.get('post', ???);

store.filter('post'); // [{...}, {...}]
store.unsaved('post'); // [{...}]

Removing data from the store

A store holds Records using Collection instances. Records can be removed from Collections several ways:

store.get('post', 1234); // {...}
store.remove('post', 1234); // {...}
store.get('post', 1234); // undefined

// SimpleStore#removeAll removes up records that match a query:
// http://api.js-data.io/js-data/latest/SimpleStore.html#removeAll
store.filter('post'); // [], oops, no posts in the store yet

const insertedPosts = store.add('post', [
  { id: 1, title: 'JSData', status: 'published' },
  { id: 2, title: 'JSDataHttp', status: 'draft' },
  { id: 3, title: 'JSDataRethinkDB', status: 'draft' },
  { id: 4, title: 'JSDataFirebase', status: 'draft' },
  { id: 5, title: 'JSDataHttp', status: 'draft' },
  { id: 6, title: 'JSDataSql', status: 'draft' }
]);

store.filter('post', { status: 'draft' }); // [{...}, {...}, {...}, {...}, {...}]
store.filter('post', { status: 'published' }); // [{...}]

store.removeAll('post', { status: 'draft' }); // [{...}, {...}, {...}, {...}, {...}]

store.filter('post', { status: 'draft' }); // []
store.filter('post', { status: 'published' }); // [{...}]

store.add('post', { id: 1234, title: 'JSData' });
store.add('comment', { id: 33, content: 'Sweet!', post_id: 1234 });

store.get('post', 1234); // {...}
store.get('comment', 33); // {...}

store.clear(); // { post: [{...}], comment: [{...}] }

store.get('post', 1234); // undefined
store.get('comment', 33); // undefined

// SimpleStore#prune removes records that do not have primary keys
// http://api.js-data.io/js-data/latest/SimpleStore.html#prune

// Insert a post that does have a primary key
store.add({ id: 1234, title: 'JSData', status: 'published' });

store.filter('post'); // [{...}]
store.unsaved('post'); // []

// Insert a post that doesn't have a primary key yet
store.add({ title: 'my new post', status: 'draft' });

store.filter('post'); // [{...}, {...}]
store.unsaved('post'); // [{...}]

store.prune('post'); // [{...}]

store.filter('post'); // [{...}]
store.unsaved('post'); // []

Listening to Events

SimpleStore and DataStore inherit from the Component class like most other JSData classes, and therefore are able to to emit events. Stores emit the events of their constituents, meaning that Mapper, Collection, and Record events bubble up to the store, so you can listen for those events right on the store.

👍

Tip

A quick way to see what's happening in your store is to print all events:

const listener = store.on('all', console.log);

// later, to stop listening
store.off(listener);

🚧

See an issue with this tutorial?

You can open an issue or better yet, suggest edits right on this page.

📘

Need support?

Have a technical question? Post on the js-data Stack Overflow channel or the Mailing list.

Want to chat with the community? Hop onto the js-data Slack channel.

Updated about 7 years ago

What’s Next

Move on to the How-To Guides or explore other documentation.