GraphQL: filtering, sorting and swapping on nested objects from separate data sources?

Question

I am trying to use graphql to combine multiple rest endpoints, and I am fixated on how to filter, sort, and output the resulting data. In particular, I need to filter and / or sort by nested values.

I cannot filter on the remaining endpoints in all cases, because they are separate microservices with separate databases. (i.e. I could filter on title at the rest of the endpoint for articles, but not on author.name). Similarly with sorting. And without filtering and sorting, pagination cannot be performed on other endpoints.

To illustrate the problem and as an attempt to solve, I came up with the following: formatResponse in apollo-server , but I'm wondering if there is a better way.

I developed a solution for the smallest set of files I could think of:

data.js represents what will be returned by two dummy rest endpoints:

 export const Authors = [{ id: 1, name: 'Sam' }, { id: 2, name: 'Pat' }]; export const Articles = [ { id: 1, title: 'Aardvarks', author: 1 }, { id: 2, title: 'Emus', author: 2 }, { id: 3, title: 'Tapir', author: 1 }, ] 

a circuit is defined as:

 import _ from 'lodash'; import { GraphQLSchema, GraphQLObjectType, GraphQLList, GraphQLString, GraphQLInt, } from 'graphql'; import { Articles, Authors, } from './data'; const AuthorType = new GraphQLObjectType({ name: 'Author', fields: { id: { type: GraphQLInt, }, name: { type: GraphQLString, } } }); const ArticleType = new GraphQLObjectType({ name: 'Article', fields: { id: { type: GraphQLInt, }, title: { type: GraphQLString, }, author: { type: AuthorType, resolve(article) { return _.find(Authors, { id: article.author }) }, } } }); const RootType = new GraphQLObjectType({ name: 'Root', fields: { articles: { type: new GraphQLList(ArticleType), resolve() { return Articles; }, } } }); export default new GraphQLSchema({ query: RootType, }); 

And the main index.js is:

 import express from 'express'; import { apolloExpress, graphiqlExpress } from 'apollo-server'; var bodyParser = require('body-parser'); import _ from 'lodash'; import rql from 'rql/query'; import rqlJS from 'rql/js-array'; import schema from './schema'; const PORT = 8888; var app = express(); function formatResponse(response, { variables }) { let data = response.data.articles; // Filter if ({}.hasOwnProperty.call(variables, 'q')) { // As an example, use a resource query lib like https://github.com/persvr/rql to do easy filtering // in production this would have to be tightened up alot data = rqlJS.query(rql.Query(variables.q), {}, data); } // Sort if ({}.hasOwnProperty.call(variables, 'sort')) { const sortKey = _.trimStart(variables.sort, '-'); data = _.sortBy(data, (element) => _.at(element, sortKey)); if (variables.sort.charAt(0) === '-') _.reverse(data); } // Pagination if ({}.hasOwnProperty.call(variables, 'offset') && variables.offset > 0) { data = _.slice(data, variables.offset); } if ({}.hasOwnProperty.call(variables, 'limit') && variables.limit > 0) { data = _.slice(data, 0, variables.limit); } return _.assign({}, response, { data: { articles: data }}); } app.use('/graphql', bodyParser.json(), apolloExpress((req) => { return { schema, formatResponse, }; })); app.use('/graphiql', graphiqlExpress({ endpointURL: '/graphql', })); app.listen( PORT, () => console.log(`GraphQL Server running at http://localhost:${PORT}`) ); 

For convenience of reference, these files are available in this meaning .

With this setting, I can send this request:

 { articles { id title author { id name } } } 

Along with these variables (it seems this is not intended to be used in variables, but this was the only way to get the post-processing parameters into the formatResponse function.):

 { "q": "author/name=Sam", "sort": "-id", "offset": 1, "limit": 1 } 

and get this answer, filter to the place where Sam is the author, sorted by descending name and get a second page where the page size is 1.

 { "data": { "articles": [ { "id": 1, "title": "Aardvarks", "author": { "id": 1, "name": "Sam" } } ] } } 

Or these variables:

 { "sort": "-author.name", "offset": 1 } 

For this answer, sort by author’s name in descending order and get all articles except the first.

 { "data": { "articles": [ { "id": 1, "title": "Aardvarks", "author": { "id": 1, "name": "Sam" } }, { "id": 2, "title": "Emus", "author": { "id": 2, "name": "Pat" } } ] } } 

So, as you can see, I use the formatResponse function for post-processing to filter / paginate / sort ..

So my questions are:

• Is this a valid use case?
• Is there a more canonical way of filtering by deeply nested properties, as well as sorting and swapping?

Answer 1

Is this a valid use case? Is there a more canonical way of filtering by deeply nested properties, as well as sorting and swapping?

The main part of the original quests is to separate the collections in different databases on separate microservices. In fact, to perform the merging of assemblies and subsequent filtering on some key, but this is impossible, since the source collection does not have a field for filtering, sorting or pagination.

The Strightforward solution performs complete or filtered queries on the source collections, and then aggregates and filters the result data set on the application server, for example. from lodash, for example, with your decision. In is possible for small collections, but in general it causes a large data transfer and uneven sorting, since there is no index structure - a real RB tree or SkipList, so with quadratic complexity this is not very good.

Depending on the amount of resources on the application server, you can create special cache tables and index tables. If the collection structure is corrected, some relations between the collection entries and their fields can be reflected in a special search table and updated accordingly on demain. This is similar to creating a search and search index, but not on the database, but on the application server. Of course, it will consume resources, but it will be faster than direct sorting by lodash.

Also, the problem can be solved on the other hand, if there is access to the structure of the source databases. The key is denormalization. In the counter, for approaching classical relations, collections may have duplicated information for subsequent combining. For example, a collection of articles may contain some information from a collection of authors, which is necessary to perform filtering, sorting, and pagination during further operations.