Index
Index proccess for system tables
Now index service is uses one _changes stream for system tables. If rules for one or more tables has changed or added, first of all need fix rules in sys.config and create reindex task for reindexing all docs from 0 sequence. Rules from sys.config
will be applied only for new cluster.
Example of task file:
Path: table_files/system_index_task.txt
Body:
[[{table_rules, {user,[{"id",{"test_temp","id",[pk,{varchar,100}]}},
{"email",{"test_temp","email",[{varchar,100}]}},
{"phone",{"test_temp","phone",[{varchar,100}]}},
{"first_name",{"test_temp","first_name",[{varchar,100}]}},
{"last_name",{"test_temp","last_name",[{varchar,100}]}}]}},
{temp_table, "test_temp"},
{table, "users"}],
[{table_rules,{gtin,[{"id",{"test_temp_gtin","id",[pk,{varchar,100}]}},
{"owner",{"test_temp_gtin","owner",[{varchar,100}]}},
{"name",{"test_temp_gtin","name",[text]}},
{"brand_id",{"test_temp_gtin","brand_id",[{varchar,100}]}}]}},
{temp_table, "test_temp_gtin"},
{table, "gtin"}
]
].
In presented example new process will be indexing table test_temp
and test_temp_gtin
from 0 sequence. After ends, table test_temp_gtin
will be replaced to gtin
and test_temp
will be replaced to users
, task file system_index_task.txt
will be deleted and new rules from file will be applied.
Get the list of supported index types
gtin
- index type of products
slgtin
- index type of serialized products
gdti
- index type of gdti documents
GET /index-service/index-schema/types
Parameters
Name |
Type |
Description |
authenticateit_identity_ticket |
header |
Required: Session's ticket |
Example Response
["gtin", "slgtin", "gdti"]
Create participant index schema
POST /index-service/index-schema
Parameters
Name |
Type |
Description |
authenticateit_identity_ticket |
header |
Required: Session's ticket |
type |
string |
index type ("gtin" , "slgtin" or "gdti" ) |
fields |
object |
{"Made In":"Made In", "Other Title":"Name column in web"} |
source_types |
array::string |
sources - ["gs1","brand","expert","retailer","contributor"] |
language |
string |
language code in ISO e.g. en, ru |
country |
string |
country code e.g. 036 |
Example Response
{
"conditions": {
"owner": "urn:authenticateit:participant:1489226097512"
},
"created_at": "2017-03-31T12:07:42Z",
"fields": {
"field_74f1be4b699c7dd0ad09d09ea7bbea7890029027": {
"Other Text Widget Title": "Name column in web"
},
"field_887c9bb54fa66e8968170597831a48ce56000ccd": {
"Made In": "Made In"
}
},
"id": "index_schema_gtin_urn:authenticateit:participant:1489226097512",
"index_type": "gtin",
"participant_id": "urn:authenticateit:participant:1489226097512",
"source_types": [
"gs1",
"brand"
],
"table": "index_schema_table_gtin_15a4ebb9-eaca-4f02-8329-4415453e6cd5",
"type": "index_schema",
"updated_at": "2017-03-31T12:07:42Z",
"user_id": "urn:authenticateit:user:email:my@gmail.com"
}
Get participant index schema
GET /index-service/index-schema
Parameters
Name |
Type |
Description |
authenticateit_identity_ticket |
header |
Required: Session's ticket |
Example Response
[
{
"conditions": {
"owner": "urn:authenticateit:participant:1489226097512"
},
"fields": {
"field_74f1be4b699c7dd0ad09d09ea7bbea7890029027": {
"Other Text Widget Title": "Name column in web"
},
"field_887c9bb54fa66e8968170597831a48ce56000ccd": {
"Made In": "Made In"
}
},
"id": "index_schema_gtin_urn:authenticateit:participant:1489226097512",
"source_types": [
"gs1",
"brand"
],
"table": "index_schema_table_gtin_15a4ebb9-eaca-4f02-8329-4415453e6cd5",
"type": "gtin"
}
]
Get index tables of ancestor brand participants
Endpoint for supply chain participants to find out SGTIN indexing
tables of supply chain ancestor brands. The response is a JSON object
with brand participant identifiers as keys. Each participant entry has
table
field (table name from participant's SGTIN/LGTIN indexing
schema).
GET /index-service/index-schema/brand-tables
Parameters
Name |
Type |
Description |
authenticateit_identity_ticket |
header |
Required: Session's ticket |
Example Response
{
"urn:authenticateit:participant:576318441844504": {
"table": "index_schema_table_slgtin_cf7372e8-40f9-4ab4-b2e0-75bd42b1a213"
}
}
Get table's profiles for RSQL
GET /index-service/rsql/profiles/:table
Parameters
Name |
Type |
Description |
authenticateit_identity_ticket |
header |
Required: Session's ticket |
table |
- |
Required: Table |
Example Response
[
{
"fields": [
"field_64fdc5235d65719db2313923ad30ce7fda154808",
"id"
],
"profile": "test_profile"
}
]
Request SQL
POST /index-service/rsql/:table
Parameters
Name |
Type |
Description |
authenticateit_identity_ticket |
header |
Required: Session's ticket |
query |
string |
rsql query |
skip |
- |
like offset in sql |
take |
- |
like limit in sql |
op |
- |
condition OR/AND |
include_scans |
- |
optional option (for data or count ) |
order |
- |
set order. Example: order=id-asc,name-desc |
type |
- |
set type of request: data , count or group |
profile |
- |
request profile (for data or count ) |
convert_date |
- |
For group type. Use for date conversion (as day, year ot month) |
group_by_fields |
- |
For group type. List of group types |
group_by_count_field |
- |
For group type. Count field name |
include_scans
- optional parameter for product table (boolean type: true or false). It is used to count user scans (the parameter user_scans
is added to the answer)
If the rsql filter value contains some special symbols (;
, "
), filter value should be quoted. For example:
id=like=|\"some value \\\" with one quote\"|
id=in=(\"Some value ; with rsql delimiter\",1,2,3);id!=123
Example rsql
id==123;name==wklnfer
id!=123;name!=wklnfer
| id=le=123;name=le=wklnfer | where id < 123 |
| id=lt=123;name=lt=wklnfer | where id ⇐ 123 |
| id=gt=123;name=gt=wklnfer | where id > 123 |
| id=ge=123;name=gt=wklnfer | where id >= 123 |
id=in=(d9683bd2-fd3e-4b0d-93bb-b58c8c2da284,1,2,3)
id=out=(d9683bd2-fd3e-4b0d-93bb-b58c8c2da284,1,2,3)
id=in=(d9683bd2-fd3e-4b0d-93bb-b58c8c2da284,1,2,3);name==test
id==123;name=like=|wklnfer| for like need set | instead of %
name=unlike=|wklnfer|
| name=null | name IS NULL |
| name=not_null | name IS NOT NULL |
Example RSQL GROUP Type Request
{
"query": "owner==urn:authenticateit:participant:804355519238230;date_created=ge=2020-10-01T00:00:00Z",
"type": "group",
"group_by_fields": "issuer;owner;date_created",
"convert_date": [
{
"field": "date_created",
"format": "day"
}
],
"group_by_count_field": "id",
"order": "date_created_day-desc"
}
Example Response
{
"count": 1,
"data": [
{
"id": "04444455368975",
"image": null,
"name": "Test product 04444455368975",
"owner": "urn:authenticateit:participant:1489513906969",
"sources": null,
"user_scans": 12
}
]
}
Request SQL. partner_brand
products filter
POST /index-service/rsql/gtin
Parameters
Name |
Type |
Description |
authenticateit_identity_ticket |
header |
Required: Session's ticket |
partner_brand |
enum |
Required: partner_brand filter value: true ,false ,null |
query |
string |
rsql query |
skip |
- |
like offset in sql |
take |
- |
like limit in sql |
op |
- |
condition OR/AND |
order |
- |
set order. Example: order=id-asc,name-desc |
type |
- |
set type of request: data |
Method supports type data
only.
query
filter should be prepared only by predefined fields name
(gtin name) and/or id
(gtin value).
partner_brand
is required and could be set as true
, false
or null
Example RSQL GROUP Type Request
{
"query": "id=like=|Wow|;name=like=|Wow|",
"op": "or",
"type": "data",
"partner_brand": true,
"take": 5
}
Example Response
[
{
"brand_id": null,
"id": "00000093524919",
"name": "WOW Select Baby Shampoo & Conditioner 200ml",
"owner": "urn:authenticateit:participant:708023102856412"
},
{
"brand_id": null,
"id": "00000093524933",
"name": "WOW Select Head to Toe Baby Wash 200ml",
"owner": "urn:authenticateit:participant:708023102856412"
},
{
"brand_id": null,
"id": "00000093524957",
"name": "WOW Select Mild Baby Shampoo 200ml",
"owner": "urn:authenticateit:participant:708023102856412"
},
{
"brand_id": null,
"id": "00000093524964",
"name": "WOW Select Calming Baby Bath 200ml",
"owner": "urn:authenticateit:participant:708023102856412"
},
{
"brand_id": null,
"id": "00000093633154",
"name": "WOW SELECT TURKEY 85G",
"owner": "urn:authenticateit:participant:708023102856412"
}
]
Clear cache
Clear cache for RSQL count requests
POST /index-service/clear_cache
Parameters
Name |
Type |
Description |
authenticateit_identity_ticket |
header |
Required: Session's ticket |
date_from |
string |
“2017-03-15T15:10:23Z” |
date_to |
string |
“2017-03-15T15:10:33Z” |
Example Response
"Cache successfully deleted"
Request API
POST /index-service/rapi
Parameters
Name |
Type |
Description |
authenticateit_identity_ticket |
header |
Required: Session's ticket |
table |
string |
table to select e.g. "gtin", "participant" or "brand" |
page |
number |
page 0,1,2... |
limit |
number |
limit per page 10,20... |
company_prefix |
string |
"9300" |
Example Response
[
{
"id": "09300633912000",
"name": "Woolworths Select 5 Sectioning Clips Silver",
"owner": "urn:authenticateit:participant:1486385901460"
},
{
"id": "09300633911959",
"name": "Woolworths Select 4 Bun Nets Light Brown",
"owner": "urn:authenticateit:participant:1486385901460"
}
]
Current system tables
authentication_result |
id |
event_ts |
user_id |
code |
country |
language |
latitude |
longitude |
gtin |
id |
owner |
name |
brand_id |
index_schema |
id |
table |
participant_id |
index_type |
user_id |
created_at |
updated_at |
participant |
id |
name |
parent |
system |
partner |
product360 |
manufacturer |
outsource |
distributor |
retailer |
expert |
supplier |
company_prefix |
rewards_campaign |
id |
owner |
name |
products |
events |
start_date |
end_date |
users |
id |
email |
language |
country |
shutout |
id |
owner |
name |
products |
start_date |
end_date |
business_location |
id |
owner |
name |
longitude |
latitude |