Getting started

Start Phalanx on a local machine using a local file system

Phalanx can be started on a local machine using a local file system as a metastore. The following command starts with a configuration file:

% ./bin/phalanx --index-metastore-uri=file:///tmp/phalanx-metastore

A metastore is a place where various information about an index is stored.

Create index on local file system

If you have started Phalanx to use the local file system, you can use this command to create an index.

% curl -XPUT -H 'Content-type: application/json' http://localhost:8000/v1/indexes/example --data-binary '
{
    "index_uri": "file:///tmp/phalanx-indexes/example",
    "index_mapping": {
        "id": {
            "type": "numeric",
            "options": {
                "index": true,
                "store": true,
                "sortable": true,
                "aggregatable": true
            }
        },
        "text": {
            "type": "text",
            "options": {
                "index": true,
                "store": true,
                "term_positions": true,
                "highlight": true,
                "sortable": true,
                "aggregatable": true
            },
            "analyzer": {
                "char_filters": [
                    {
                        "name": "ascii_folding"
                    },
                    {
                        "name": "unicode_normalize",
                        "options": {
                            "form": "NFKC"
                        }
                    }
                ],
                "tokenizer": {
                    "name": "unicode"
                },
                "token_filters": [
                    {
                        "name": "lower_case"
                    }
                ]
            }
        }
    },
    "num_shards": 1,
    "default_search_field": "_all",
    "default_analyzer": {
        "tokenizer": {
            "name": "unicode"
        },
        "token_filters": [
            {
                "name": "lower_case"
            }
        ]
    }
}
'

Health check

These endpoints should be used for Phalanx health checks.

Liveness check

If Phalanx is running properly, it will return HTTP status 200.

% curl -XGET http://localhost:8000/livez | jq .
{
  "state":"alive"
}

Readiness check

If Phalanx is ready to accept the traffic, it will return HTTP Status 200.

% curl -XGET http://localhost:8000/readyz | jq .
{
  "state":"ready"
}

But this endpoint is not yet fully implemented.

Metrics exposition

This endpoint returns Phalanx metrics in Prometheus exposition format.

% curl -XGET http://localhost:8000/metrics
# HELP phalanx_grpc_server_handled_total Total number of RPCs completed on the server, regardless of success or failure.
# TYPE phalanx_grpc_server_handled_total counter
phalanx_grpc_server_handled_total{grpc_code="Aborted",grpc_method="AddDocuments",grpc_service="index.Index",grpc_type="unary"} 0
phalanx_grpc_server_handled_total{grpc_code="Aborted",grpc_method="Cluster",grpc_service="index.Index",grpc_type="unary"} 0
...
phalanx_grpc_server_started_total{grpc_method="Metrics",grpc_service="index.Index",grpc_type="unary"} 1
phalanx_grpc_server_started_total{grpc_method="ReadinessCheck",grpc_service="index.Index",grpc_type="unary"} 0
phalanx_grpc_server_started_total{grpc_method="Search",grpc_service="index.Index",grpc_type="unary"} 0

Cluster status

This endpoint returns the latest cluster status.

  • nodes: Lists the nodes that are joining in the cluster.
  • indexes: Lists the indexes served by the cluster.
  • indexer_assignment: Lists which node is responsible for the shard in the index.
  • searcher_assignment: Lists which nodes are responsible for the shard in the index.
% curl -XGET http://localhost:8000/cluster | jq .
{
  "indexer_assignment": {
    "wikipedia_en": {
      "shard-73iAEf8K": "node-duIMwfjn",
      "shard-CRzZVi2b": "node-duIMwfjn",
      "shard-Wh7VO5Lp": "node-duIMwfjn",
      "shard-YazeIhze": "node-duIMwfjn",
      "shard-cXyt4esz": "node-duIMwfjn",
      "shard-hUM3HWQW": "node-duIMwfjn",
      "shard-jH3sTtc7": "node-duIMwfjn",
      "shard-viI2Dm3V": "node-duIMwfjn",
      "shard-y1tMwCEP": "node-duIMwfjn",
      "shard-y7VRCIlU": "node-duIMwfjn"
    }
  },
  "indexes": {
    "wikipedia_en": {
      "index_lock_uri": "",
      "index_uri": "file:///tmp/phalanx/indexes/wikipedia_en",
      "shards": {
        "shard-73iAEf8K": {
          "shard_lock_uri": "",
          "shard_uri": "file:///tmp/phalanx/indexes/wikipedia_en/shard-73iAEf8K"
        },
        "shard-CRzZVi2b": {
          "shard_lock_uri": "",
          "shard_uri": "file:///tmp/phalanx/indexes/wikipedia_en/shard-CRzZVi2b"
        },
        "shard-Wh7VO5Lp": {
          "shard_lock_uri": "",
          "shard_uri": "file:///tmp/phalanx/indexes/wikipedia_en/shard-Wh7VO5Lp"
        },
        "shard-YazeIhze": {
          "shard_lock_uri": "",
          "shard_uri": "file:///tmp/phalanx/indexes/wikipedia_en/shard-YazeIhze"
        },
        "shard-cXyt4esz": {
          "shard_lock_uri": "",
          "shard_uri": "file:///tmp/phalanx/indexes/wikipedia_en/shard-cXyt4esz"
        },
        "shard-hUM3HWQW": {
          "shard_lock_uri": "",
          "shard_uri": "file:///tmp/phalanx/indexes/wikipedia_en/shard-hUM3HWQW"
        },
        "shard-jH3sTtc7": {
          "shard_lock_uri": "",
          "shard_uri": "file:///tmp/phalanx/indexes/wikipedia_en/shard-jH3sTtc7"
        },
        "shard-viI2Dm3V": {
          "shard_lock_uri": "",
          "shard_uri": "file:///tmp/phalanx/indexes/wikipedia_en/shard-viI2Dm3V"
        },
        "shard-y1tMwCEP": {
          "shard_lock_uri": "",
          "shard_uri": "file:///tmp/phalanx/indexes/wikipedia_en/shard-y1tMwCEP"
        },
        "shard-y7VRCIlU": {
          "shard_lock_uri": "",
          "shard_uri": "file:///tmp/phalanx/indexes/wikipedia_en/shard-y7VRCIlU"
        }
      }
    }
  },
  "nodes": {
    "node-duIMwfjn": {
      "addr": "0.0.0.0",
      "meta": {
        "grpc_port": 5000,
        "http_port": 8000,
        "roles": [
          "indexer",
          "searcher"
        ]
      },
      "port": 3000,
      "state": "alive"
    }
  },
  "searcher_assignment": {
    "wikipedia_en": {
      "shard-73iAEf8K": [
        "node-duIMwfjn"
      ],
      "shard-CRzZVi2b": [
        "node-duIMwfjn"
      ],
      "shard-Wh7VO5Lp": [
        "node-duIMwfjn"
      ],
      "shard-YazeIhze": [
        "node-duIMwfjn"
      ],
      "shard-cXyt4esz": [
        "node-duIMwfjn"
      ],
      "shard-hUM3HWQW": [
        "node-duIMwfjn"
      ],
      "shard-jH3sTtc7": [
        "node-duIMwfjn"
      ],
      "shard-viI2Dm3V": [
        "node-duIMwfjn"
      ],
      "shard-y1tMwCEP": [
        "node-duIMwfjn"
      ],
      "shard-y7VRCIlU": [
        "node-duIMwfjn"
      ]
    }
  }
}

Add / Update documents

% curl -XPUT -H 'Content-type: application/x-ndjson' http://localhost:8000/v1/indexes/example/documents --data-binary '
{"_id":"1", "id":1, "text":"This is an example document 1."}
{"_id":"2", "id":2, "text":"This is an example document 2."}
{"_id":"3", "id":3, "text":"This is an example document 3."}
'

Delete documents

% curl -XDELETE -H 'Content-type: text/plain' http://localhost:8000/v1/indexes/example/documents --data-binary '
1
2
3
'
% curl -XPOST -H 'Content-type: application/json' http://localhost:8000/v1/indexes/example/_search --data-binary '
{
    "query": "text:document",
    "boost": 1.0,
    "start": 0,
    "num": 10,
    "sort_by": "-_score",
    "fields": [
        "id",
        "text"
    ],
    "aggregations": {
        "timestamp_date_range": {
            "type": "date_range",
            "options": {
                "field": "_timestamp",
                "ranges": {
                    "year_before_last": {
                        "start": "2020-01-01T00:00:00Z",
                        "end": "2021-01-01T00:00:00Z"
                    },
                    "last_year": {
                        "start": "2021-01-01T00:00:00Z",
                        "end": "2022-01-01T00:00:00Z"
                    },
                    "this_year": {
                        "start": "2022-01-01T00:00:00Z",
                        "end": "2023-01-01T00:00:00Z"
                    }
                }
            }
        }
    }
}
'
{
  "aggregations": {
    "timestamp_date_range": {
      "last_year": 0,
      "this_year": 3,
      "year_before_last": 0
    }
  },
  "documents": [
    {
      "fields": {
        "id": 1,
        "text": "This is an example document 1."
      },
      "id": "1",
      "score": 0.06069608755660118,
      "timestamp": 1641992086513383200
    },
    {
      "fields": {
        "id": 2,
        "text": "This is an example document 2."
      },
      "id": "2",
      "score": 0.06069608755660118,
      "timestamp": 1641992086513395500
    },
    {
      "fields": {
        "id": 3,
        "text": "This is an example document 3."
      },
      "id": "3",
      "score": 0.06069608755660118,
      "timestamp": 1641992086513399600
    }
  ],
  "hits": 3,
  "index_name": "example"
}

Delete index

The following command will delete the index wikipedia_en with the specified name. This command will delete the index file on the object storage and the index metadata on the metastore.

% curl -XDELETE http://localhost:8000/v1/indexes/example

results matching ""

    No results matching ""