Index

Index proccess for system tables

Now index service is uses one _changes stream for system tables. If rules for one or more tables has changed or added, first of all need fix rules in sys.config and create reindex task for reindexing all docs from 0 sequence. Rules from sys.config will be applied only for new cluster.

Example of task file:

Path: table_files/system_index_task.txt

Body:

[[{table_rules, {user,[{"id",{"test_temp","id",[pk,{varchar,100}]}},
        {"email",{"test_temp","email",[{varchar,100}]}},
        {"phone",{"test_temp","phone",[{varchar,100}]}},
        {"first_name",{"test_temp","first_name",[{varchar,100}]}},
        {"last_name",{"test_temp","last_name",[{varchar,100}]}}]}},
  {temp_table, "test_temp"},
  {table, "users"}],
  [{table_rules,{gtin,[{"id",{"test_temp_gtin","id",[pk,{varchar,100}]}},
        {"owner",{"test_temp_gtin","owner",[{varchar,100}]}},
        {"name",{"test_temp_gtin","name",[text]}},
        {"brand_id",{"test_temp_gtin","brand_id",[{varchar,100}]}}]}},
    {temp_table, "test_temp_gtin"},
    {table, "gtin"}
    ]
  ].

In presented example new process will be indexing table test_temp and test_temp_gtin from 0 sequence. After ends, table test_temp_gtin will be replaced to gtin and test_temp will be replaced to users, task file system_index_task.txt will be deleted and new rules from file will be applied.

Get the list of supported index types

gtin - index type of products slgtin - index type of serialized products gdti - index type of gdti documents

GET /index-service/index-schema/types
Parameters
Name Type Description
authenticateit_identity_ticket header Required: Session's ticket
Example Response
["gtin", "slgtin", "gdti"]

Create participant index schema

POST /index-service/index-schema
Parameters
Name Type Description
authenticateit_identity_ticket header Required: Session's ticket
type string index type ("gtin", "slgtin" or "gdti")
fields object {"Made In":"Made In", "Other Title":"Name column in web"}
link_fields list Link index rules
source_types array::string sources - ["gs1","brand","expert","retailer","contributor"]
language string language code in ISO e.g. en, ru
country string country code e.g. 036
Example Request
{
  "conditions": {
    "owner": "urn:authenticateit:participant:735879621218609"
  },
  "fields": {
    "field_0e76292794888d4f1fa75fb3aff4ca27c58f56a6": {
      "image": "Image"
    },
    "field_3c6de1b7dd91465d437ef415f94f36afc1fbc8a8": {
      "title": "Title"
    },
    "field_414f505575623b99b6fe418814b146c3f52f1bd4": {
      "serial_number": "Serial Number"
    },
    "field_579233b2c479241523cba5e3af55d0f50f2d6414": {
      "owner": "Owner"
    },
    "field_5bb1d40aad5728573e7b5d2db72c584057d74170": {
      "company_prefix": "Company Prefix"
    },
    "field_87ea5dfc8b8e384d848979496e706390b497e547": {
      "id": "ID"
    },
    "field_b584e85a40e6e2574fddf520398df3604a66fb9a": {
      "doc_type": "Document Type"
    }
  },
  "id": "index_schema_gdti_urn:authenticateit:participant:735879621218609",
  "source_types": [
    "brand",
    "contributor",
    "expert",
    "gs1",
    "retailer",
    "skuvantage"
  ],
  "link_fields": [
    {
      "text": "Link Text Str",
      "image_field_name": "Web Column name img",
      "url_field_name": "Web Column name url"
    }
  ],
  "type": "gdti"
}
Example Response
{
    "conditions": {
        "owner": "urn:authenticateit:participant:735879621218609"
    },
    "created_at": "2019-04-15T03:52:52Z",
    "default_fields": {
        "company_prefix": {
            "company_prefix": "Company Prefix"
        },
        "doc_type": {
            "doc_type": "Document Type"
        },
        "id": {
            "id": "ID"
        },
        "image": {
            "image": "Image"
        },
        "owner": {
            "owner": "Owner"
        },
        "serial_number": {
            "serial_number": "Serial Component"
        },
        "title": {
            "title": "Title"
        }
    },
    "fields": {
        "field_0e76292794888d4f1fa75fb3aff4ca27c58f56a6": {
            "image": "Image"
        },
        "field_3c6de1b7dd91465d437ef415f94f36afc1fbc8a8": {
            "title": "Title"
        },
        "field_414f505575623b99b6fe418814b146c3f52f1bd4": {
            "serial_number": "Serial Number"
        },
        "field_579233b2c479241523cba5e3af55d0f50f2d6414": {
            "owner": "Owner"
        },
        "field_5bb1d40aad5728573e7b5d2db72c584057d74170": {
            "company_prefix": "Company Prefix"
        },
        "field_87ea5dfc8b8e384d848979496e706390b497e547": {
            "id": "ID"
        },
        "field_b584e85a40e6e2574fddf520398df3604a66fb9a": {
            "doc_type": "Document Type"
        }
    },
    "id": "index_schema_gdti_urn:authenticateit:participant:735879621218609",
    "index_type": "gdti",
    "link_fields": [
        {
            "image_field": "field_2ba36518745e34c4a06b945f1e050cea3af0b59c",
            "image_field_name": "Web Column name img",
            "text": "link text str",
            "url_field": "field_41e726918505fd70292de8a204cf3093f8743350",
            "url_field_name": "Web Column name url"
        }
    ],
    "participant_id": "urn:authenticateit:participant:735879621218609",
    "source_types": [
        "brand",
        "contributor",
        "expert",
        "gs1",
        "retailer",
        "skuvantage"
    ],
    "table": "index_schema_table_gdti_cc268bc2-63cb-480c-8193-3766b6d67e3b",
    "type": "index_schema",
    "updated_at": "2021-04-01T15:08:14Z",
    "user_id": "urn:authenticateit:user:email:ethertest3@mailinator.com"
}

Get participant index schema

GET /index-service/index-schema
Parameters
Name Type Description
authenticateit_identity_ticket header Required: Session's ticket
Example Response
[
    {
        "conditions": {
            "owner": "urn:authenticateit:participant:735879621218609"
        },
        "default_fields": {
            "company_prefix": {
                "company_prefix": "Company Prefix"
            },
            "doc_type": {
                "doc_type": "Document Type"
            },
            "id": {
                "id": "ID"
            }
        },
        "fields": {
            "field_87ea5dfc8b8e384d848979496e706390b497e547": {
                "id": "ID"
            },
            "field_b584e85a40e6e2574fddf520398df3604a66fb9a": {
                "doc_type": "Document Type"
            }
        },
        "link_fields": [
            {
                "image_field": "field_2ba36518745e34c4a06b945f1e050cea3af0b59c",
                "image_field_name": "Web Column name img",
                "text": "link text str",
                "url_field": "field_41e726918505fd70292de8a204cf3093f8743350",
                "url_field_name": "Web Column name url"
            }
        ],
        "id": "index_schema_gdti_urn:authenticateit:participant:735879621218609",
        "source_types": [
            "brand",
            "contributor",
            "expert",
            "gs1",
            "retailer",
            "skuvantage"
        ],
        "table": "index_schema_table_gdti_cc268bc2-63cb-480c-8193-3766b6d67e3b",
        "type": "gdti"
    },
    {
        "conditions": {
            "custody": "urn:authenticateit:participant:735879621218609",
            "owner": "urn:authenticateit:participant:735879621218609"
        },
        "default_fields": {
            "atk_sscc": {
                "atk_sscc": "ATK SSCC"
            },
            "num_contributor_sources": {
                "num_contributor_sources": "Contributor Sources"
            },
            "atk_gs1_sscc": {
                "atk_gs1_sscc": "ATK GS1 SSCC"
            },
            "code": {
                "code": "GS1 Databar"
            },
            "image": {
                "image": "Image"
            },
            "task": {
                "task": "Serialization Task"
            },
            "delivery_number": {
                "delivery_number": "Delivery"
            },
            "user_reviews": {
                "user_reviews": "Reviews"
            },
            "id": {
                "id": "ID"
            }
        },
        "fields": {},
        "id": "index_schema_slgtin_urn:authenticateit:participant:735879621218609",
        "source_types": [
            "brand",
            "contributor",
            "expert",
            "gs1",
            "retailer"
        ],
        "table": "index_schema_table_slgtin_07a62602-6cc9-430f-bd28-cf14d9747ac9",
        "type": "slgtin"
    },
    {
        "conditions": {
            "owner": "urn:authenticateit:participant:735879621218609"
        },
        "default_fields": {
            "brand_id": {
                "brand_id": "Brand"
            },
            "gpc_brick": {
                "gpc_brick": "GPC Brick"
            },
            "gpc_class": {
                "gpc_class": "GPC Class"
            },
            "gpc_family": {
                "gpc_family": "GPC Family"
            },
            "gpc_segment": {
                "gpc_segment": "GPC Segment"
            },
            "id": {
                "id": "ID"
            }
        },
        "fields": {},
        "id": "index_schema_gtin_urn:authenticateit:participant:735879621218609",
        "source_types": [
            "gs1",
            "brand",
            "expert",
            "retailer",
            "contributor"
        ],
        "table": "index_schema_table_gtin_735879621218609",
        "type": "gtin"
    }
]

Create manual reindex task

POST /index-service/index-schema/reindex
Parameters
Name Type Description
authenticateit_identity_ticket header Required: Session's ticket
type string Required: Index type (gtin, slgtin or gdti)
Example Request
{
  "type": "gdti"
}
Example Response
{
    "id": "urn:authenticateit:participant:735879621218609:gdti:reindex"
}

Read manual reindex task status

GET /index-service/index-schema/reindex
Parameters
Name Type Description
authenticateit_identity_ticket header Required: Session's ticket
id string Required: Task Id
Example Response
{
    "key": "urn:authenticateit:participant:735879621218609:gdti:reindex",
    "new_table": "gdti_ee259a99-f830-4972-8abf-59b12c11658f_reindex",
    "original_table": "index_schema_table_gdti_cc268bc2-63cb-480c-8193-3766b6d67e3b",
    "owner": "urn:authenticateit:participant:735879621218609",
    "pid": "<0.28840.0>",
    "progress": 7,
    "status": "done"
}

Get index tables of ancestor brand participants

Endpoint for supply chain participants to find out SGTIN indexing tables of supply chain ancestor brands. The response is a JSON object with brand participant identifiers as keys. Each participant entry has table field (table name from participant's SGTIN/LGTIN indexing schema).

GET /index-service/index-schema/brand-tables
Parameters
Name Type Description
authenticateit_identity_ticket header Required: Session's ticket
Example Response
{
  "urn:authenticateit:participant:576318441844504": {
    "table": "index_schema_table_slgtin_cf7372e8-40f9-4ab4-b2e0-75bd42b1a213"
  }
}

Get table's profiles for RSQL

GET /index-service/rsql/profiles/:table
Parameters
Name Type Description
authenticateit_identity_ticket header Required: Session's ticket
table - Required: Table
Example Response
[
    {
        "fields": [
            "field_64fdc5235d65719db2313923ad30ce7fda154808",
            "id"
        ],
        "profile": "test_profile"
    }
]

Request SQL

POST /index-service/rsql/:table
Parameters
Name Type Description
authenticateit_identity_ticket header Required: Session's ticket
query string rsql query
skip - like offset in sql
take - like limit in sql
op - condition OR/AND
include_scans - optional option (for data or count)
order - set order. Example: order=id-asc,name-desc
type - set type of request: data, count or group
profile - request profile (for data or count)
convert_date - For group type. Use for date conversion (as day, year ot month)
group_by_fields - For group type. List of group types
group_by_count_field - For group type. Count field name

include_scans - optional parameter for product table (boolean type: true or false). It is used to count user scans (the parameter user_scans is added to the answer)

If the rsql filter value contains some special symbols (;, "), filter value should be quoted. For example:
id=like=|\"some value \\\" with one quote\"|
id=in=(\"Some value ; with rsql delimiter\",1,2,3);id!=123

Example rsql
id==123;name==wklnfer
id!=123;name!=wklnfer
| id=le=123;name=le=wklnfer | where id < 123  |
| id=lt=123;name=lt=wklnfer | where id ⇐ 123  |
| id=gt=123;name=gt=wklnfer | where id > 123  |
| id=ge=123;name=gt=wklnfer | where id >= 123 |
id=in=(d9683bd2-fd3e-4b0d-93bb-b58c8c2da284,1,2,3)
id=out=(d9683bd2-fd3e-4b0d-93bb-b58c8c2da284,1,2,3)
id=in=(d9683bd2-fd3e-4b0d-93bb-b58c8c2da284,1,2,3);name==test
id==123;name=like=|wklnfer| for like need set | instead of %
name=unlike=|wklnfer|
| name=null     | name IS NULL      |
| name=not_null | name IS NOT NULL  |
Example RSQL GROUP Type Request
{
  "query": "owner==urn:authenticateit:participant:804355519238230;date_created=ge=2020-10-01T00:00:00Z",
  "type": "group",
  "group_by_fields": "issuer;owner;date_created",
  "convert_date": [
    {
      "field": "date_created",
      "format": "day"
    }
  ],
  "group_by_count_field": "id",
  "order": "date_created_day-desc"
}
Example Response
{
  "count": 1,
  "data": [
    {
      "id": "04444455368975",
      "image": null,
      "name": "Test product 04444455368975",
      "owner": "urn:authenticateit:participant:1489513906969",
      "sources": null,
      "user_scans": 12
    }
  ]
}

Request SQL. partner_brand products filter

POST /index-service/rsql/gtin
Parameters
Name Type Description
authenticateit_identity_ticket header Required: Session's ticket
partner_brand enum Required: partner_brand filter value: true,false,null
query string rsql query
skip - like offset in sql
take - like limit in sql
op - condition OR/AND
order - set order. Example: order=id-asc,name-desc
type - set type of request: data

Method supports type data only.
query filter should be prepared only by predefined fields name (gtin name) and/or id (gtin value).
partner_brand is required and could be set as true, false or null

Example RSQL GROUP Type Request
{
  "query": "id=like=|Wow|;name=like=|Wow|",
  "op": "or",
  "type": "data",
  "partner_brand": true,
  "take": 5
}
Example Response
[
    {
        "brand_id": null,
        "id": "00000093524919",
        "name": "WOW Select Baby Shampoo & Conditioner 200ml",
        "owner": "urn:authenticateit:participant:708023102856412"
    },
    {
        "brand_id": null,
        "id": "00000093524933",
        "name": "WOW Select Head to Toe Baby Wash 200ml",
        "owner": "urn:authenticateit:participant:708023102856412"
    },
    {
        "brand_id": null,
        "id": "00000093524957",
        "name": "WOW Select Mild Baby Shampoo 200ml",
        "owner": "urn:authenticateit:participant:708023102856412"
    },
    {
        "brand_id": null,
        "id": "00000093524964",
        "name": "WOW Select Calming Baby Bath 200ml",
        "owner": "urn:authenticateit:participant:708023102856412"
    },
    {
        "brand_id": null,
        "id": "00000093633154",
        "name": "WOW SELECT TURKEY 85G",
        "owner": "urn:authenticateit:participant:708023102856412"
    }
]

Clear cache

Clear cache for RSQL count requests

POST /index-service/clear_cache
Parameters
Name Type Description
authenticateit_identity_ticket header Required: Session's ticket
date_from string “2017-03-15T15:10:23Z”
date_to string “2017-03-15T15:10:33Z”
Example Response
"Cache successfully deleted"

Request API

POST /index-service/rapi
Parameters
Name Type Description
authenticateit_identity_ticket header Required: Session's ticket
table string table to select e.g. "gtin", "participant" or "brand"
page number page 0,1,2...
limit number limit per page 10,20...
company_prefix string "9300"
Example Response
[
  {
    "id": "09300633912000",
    "name": "Woolworths Select 5 Sectioning Clips Silver",
    "owner": "urn:authenticateit:participant:1486385901460"
  },
  {
    "id": "09300633911959",
    "name": "Woolworths Select 4 Bun Nets Light Brown",
    "owner": "urn:authenticateit:participant:1486385901460"
  }
]

Current system tables

authentication_result
id
event_ts
user_id
code
country
language
latitude
longitude
gtin
id
owner
name
brand_id
index_schema
id
table
participant_id
index_type
user_id
created_at
updated_at
participant
id
name
parent
system
partner
product360
manufacturer
outsource
distributor
retailer
expert
supplier
company_prefix
reward_point_claim
id
rewards_campaign
id
owner
name
products
events
start_date
end_date
users
id
email
language
country
shutout
id
owner
name
products
start_date
end_date
business_location
id
owner
name
longitude
latitude

results matching ""

    No results matching ""