INFINI Easysearch
INFINI Easysearchは分散型のニアリアルタイム検索・分析エンジンで、コアエンジンはオープンソースのApache Luceneをベースにしています。EasysearchはオープンソースライセンスApache 2.0に基づくElasticsearch 7.10バージョンから派生し、より多くのエンタープライズ機能をサポート・改善し、検索ビジネスシナリオを最適化することで、より優れたデータ探索と分析体験を保証します。
Easysearchの主な特徴：
- Elasticsearchとの互換性あり、ビジネスコードの変更不要。スムーズなマイグレーションが可能
- エンタープライズレベルのセキュリティ。より安定性と信頼性
- エンタープライズ管理バックエンド（マルチクラスタの一元管理による運営の標準化・自動化）
- 信創適応（国内CPU/OS/サーバーへの完全対応、国密アルゴリズムのサポート）
- Easysearchのクイックスタートについては以下をご参照ください：
Easysearch、業界最強、他に類をなしません！
Loadgen
LoadgenはElasticsearch専用の負荷テストツールで、Easysearchまたはゲートウェイに対する負荷テストに使用されます。
Loadgenの特徴：
- 高性能
- 軽量で依存関係なし
- テンプレート化されたパラメータのランダムサポート
- 高度な同時実行性のサポート
- 負荷テスト側のトラフィック制御
- サーバー側の戻り値検証のサポート
現在の最新バージョンは1.26.1-598で、ダウンロードリンクは以下の通りです：
https://release.infinilabs.com/loadgen/stable/
ダウンロードして解凍後、バイナリファイルを取得します。
バージョン情報を確認します。
$ ./loadgen-linux-amd64 -v
__ ___ _ ___ ___ __ __
/ / /___\/_\ / \/ _ \ /__\/\ \ \
/ / // ///_\\ / /\ / /_\//_\ / \/ /
/ /__/ \_// _ \/ /_// /_\\//__/ /\ /
\____|___/\_/ \_/___,'\____/\__/\_\ \/
[LOADGEN] A http load generator and testing suite.
[LOADGEN] 1.26.1#598, 2024-08-14 04:50:03, 2024-12-31 10:10:10, 00f15fd86834d7ea539f6d66ee608e3915eef0e3
loadgen 1.26.1 598 2024-08-14 04:50:03 +0000 UTC 2024-12-31 10:10:10 +0000 UTC 00f15fd86834d7ea539f6d66ee608e3915eef0e3
Loadgenテスト
Loadgenの使用は非常に簡単です。以下にクエリ負荷テストの実施方法を示します。
- インデックスを作成し、ノード数に応じてシャード数を調整します。
curl -X PUT -H "Content-Type: application/json" \
-ku admin:xxx "https://localhost:9200/performance_test" -d'
{
"settings":{
"number_of_shards":"4",
"number_of_replicas":1,
"translog":{
"durability":"async"
}
}
}'
{"acknowledged":true,"shards_acknowledged":true,"index":"performance_test"}
test_query.dsl.configという設定ファイルを作成し、クエリを定義します
GET performance_test/_search
{"query": {"term": {"document_id.keyword": {"value": $[[doc_id]]}}}}
- 負荷テストを実行します。
-dは秒数、-cは同時実行数を示します。
$ ES_ENDPOINT=https://localhost:9200 ES_USERNAME=admin ES_PASSWORD=xxx ./loadgen-linux-amd64 -run test_query.dsl.config -d 15 -c 3
__ ___ _ ___ ___ __ __
/ / /___\/_\ / \/ _ \ /__\/\ \ \
/ / // ///_\\ / /\ / /_\//_\ / \/ /
/ /__/ \_// _ \/ /_// /_\\//__/ /\ /
\____|___/\_/ \_/___,'\____/\__/\_\ \/
[LOADGEN] A http load generator and testing suite.
[LOADGEN] 1.26.1#598, 2024-08-14 04:50:03, 2024-12-31 10:10:10, 00f15fd86834d7ea539f6d66ee608e3915eef0e3
[08-19 16:43:27] [INF] [env.go:179] configuration auto reload enabled
[08-19 16:43:27] [INF] [app.go:285] initializing loadgen, pid: 30792
[08-19 16:43:27] [INF] [app.go:286] using config: /data/es/loadgen.yml
[08-19 16:43:27] [INF] [module.go:155] started plugin: statsd
[08-19 16:43:27] [INF] [module.go:161] all modules are started
[08-19 16:43:27] [INF] [instance.go:78] workspace: /data/es/data/loadgen/nodes/cr1gabqr90cng685o2s0
[08-19 16:43:27] [INF] [app.go:511] loadgen is up and running now.
[08-19 16:43:27] [INF] [main.go:403] loading config: /data/es/test_query.dsl.config
31662 requests finished in 14.9853211s, 1.62MB sent, 4.83MB received
[Loadgen Client Metrics]
Requests/sec: 2114.28
Request Traffic/sec: 108.31KB
Total Transfer/sec: 431.23KB
Fastest Request: 720.456µs
Slowest Request: 15.623821ms
Status 200: 31662
[Latency Metrics]
15000 samples of 31662 events
Cumulative: 13.2458712s
HMean: 845.231µs
Avg.: 881.45µs
p50: 828.765µs
p75: 932.123µs
p95: 1.087654ms
p99: 1.382109ms
p999: 5.12345ms
Long 5%: 1.456789ms
Short 5%: 735.678µs
Max: 15.234567ms
Min: 720.456µs
Range: 14.514111ms
StdDev: 301.234µs
Rate/sec.: 2114.28
[Latency Distribution]
720µs - 1.823ms ------------------------------
1.823ms - 2.926ms -
2.926ms - 4.029ms -
4.029ms - 5.132ms -
5.132ms - 6.235ms -
6.235ms - 7.338ms -
7.338ms - 8.441ms -
8.441ms - 9.544ms -
9.544ms - 10.647ms -
10.647ms - 15.234ms -
[Estimated Server Metrics]
Requests/sec: 2161.34
Avg Req Time: 945.123µs
Transfer/sec: 438.76KB
詳細なドキュメントは以下をご参照ください：
https://infinilabs.cn/docs/latest/gateway/getting-started/benchmark/
Rally
Elasticsearchに慣れている場合は、Rallyを使用してEasysearchのテストも試してみてください。
RallyはElasticsearchのベンチマークフレームワークです。以下のタスクを支援できます：
- ベンチマーク用のElasticsearchクラスタのインストール
- 異なるElasticsearchバージョン間でのベンチマークデータと仕様の管理
- ベンチマークの実行と結果の記録
- テレメトリデバイスへの接続によるパフォーマンス問題の特定
- パフォーマンス結果の比較
Rallyをインストールします。
[root@es-test /]# pip3 install esrally
Requirement already satisfied: esrally in /usr/local/lib/python3.9/site-packages (2.11.0)
Requirement already satisfied: py-cpuinfo==7.0.0 in /usr/local/lib/python3.9/site-packages (from esrally) (7.0.0)
Requirement already satisfied: elastic-transport==8.4.1 in /usr/local/lib/python3.9/site-packages (from esrally) (8.4.1)
Requirement already satisfied: google-resumable-media[requests]==1.1.0 in /usr/local/lib/python3.9/site-packages (from esrally) (1.1.0)
Requirement already satisfied: urllib3==1.26.18 in /usr/local/lib/python3.9/site-packages (from esrally) (1.26.18)
Requirement already satisfied: google-auth==1.22.1 in /usr/local/lib/python3.9/site-packages (from esrally) (1.22.1)
Requirement already satisfied: thespian==3.10.1 in /usr/local/lib/python3.9/site-packages (from esrally) (3.10.1)
Requirement already satisfied: jsonschema==3.1.1 in /usr/local/lib/python3.9/site-packages (from esrally) (3.1.1)
Requirement already satisfied: markupsafe==2.0.1 in /usr/local/lib64/python3.9/site-packages (from esrally) (2.0.1)
Requirement already satisfied: yappi==1.5.1 in /usr/local/lib64/python3.9/site-packages (from esrally) (1.5.1)
Requirement already satisfied: psutil==5.9.4 in /usr/local/lib64/python3.9/site-packages (from esrally) (5.9.4)
Requirement already satisfied: certifi in /usr/local/lib/python3.9/site-packages (from esrally) (2024.7.4)
Requirement already satisfied: elasticsearch[async]==8.6.1 in /usr/local/lib/python3.9/site-packages (from esrally) (8.6.1)
Requirement already satisfied: ijson==2.6.1 in /usr/local/lib/python3.9/site-packages (from esrally) (2.6.1)
Requirement already satisfied: jinja2==3.1.4 in /usr/local/lib/python3.9/site-packages (from esrally) (3.1.4)
Requirement already satisfied: requests<2.32.0 in /usr/local/lib/python3.9/site-packages (from esrally) (2.31.0)
Requirement already satisfied: tabulate==0.8.9 in /usr/local/lib/python3.9/site-packages (from esrally) (0.8.9)
Requirement already satisfied: zstandard==0.21.0 in /usr/local/lib64/python3.9/site-packages (from esrally) (0.21.0)
Requirement already satisfied: docker==6.0.0 in /usr/local/lib/python3.9/site-packages (from esrally) (6.0.0)
Requirement already satisfied: websocket-client>=0.32.0 in /usr/local/lib/python3.9/site-packages (from docker==6.0.0->esrally) (1.8.0)
Requirement already satisfied: packaging>=14.0 in /usr/lib/python3.9/site-packages (from docker==6.0.0->esrally) (20.9)
Requirement already satisfied: aiohttp<4,>=3 in /usr/local/lib64/python3.9/site-packages (from elasticsearch[async]==8.6.1->esrally) (3.10.4)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.9/site-packages (from google-auth==1.22.1->esrally) (0.4.0)
Requirement already satisfied: six>=1.9.0 in /usr/lib/python3.9/site-packages (from google-auth==1.22.1->esrally) (1.15.0)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.9/site-packages (from google-auth==1.22.1->esrally) (4.9)
Requirement already satisfied: setuptools>=40.3.0 in /usr/lib/python3.9/site-packages (from google-auth==1.22.1->esrally) (53.0.0)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.9/site-packages (from google-auth==1.22.1->esrally) (4.2.4)
Requirement already satisfied: google-crc32c<2.0dev,>=1.0 in /usr/local/lib64/python3.9/site-packages (from google-resumable-media[requests]==1.1.0->esrally) (1.5.0)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.9/site-packages (from jsonschema==3.1.1->esrally) (24.2.0)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.9/site-packages (from jsonschema==3.1.1->esrally) (8.2.0)
Requirement already satisfied: pyrsistent>=0.14.0 in /usr/local/lib64/python3.9/site-packages (from jsonschema==3.1.1->esrally) (20.0)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib64/python3.9/site-packages (from aiohttp<4,>=3->elasticsearch[async]==8.6.1->esrally) (1.9.4)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib64/python3.9/site-packages (from aiohttp<4,>=3->elasticsearch[async]==8.6.1->esrally) (1.4.1)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.9/site-packages (from aiohttp<4,>=3->elasticsearch[async]==8.6.1->esrally) (1.3.1)
Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.9/site-packages (from aiohttp<4,>=3->elasticsearch[async]==8.6.1->esrally) (4.0.3)
Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /usr/local/lib/python3.9/site-packages (from aiohttp<4,>=3->elasticsearch[async]==8.6.1->esrally) (2.3.7)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib64/python3.9/site-packages (from aiohttp<4,>=3->elasticsearch[async]==8.6.1->esrally) (6.0.5)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/lib/python3.9/site-packages (from packaging>=14.0->docker==6.0.0->esrally) (2.4.7)
Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /usr/local/lib/python3.9/site-packages (from pyasn1-modules>=0.2.1->google-auth==1.22.1->esrally) (0.6.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.9/site-packages (from requests<2.32.0->esrally) (3.7)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib64/python3.9/site-packages (from requests<2.32.0->esrally) (3.3.2)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.9/site-packages (from importlib-metadata->jsonschema==3.1.1->esrally) (3.20.0)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[root@es-test /]# esrally --version
esrally 2.11.0
[root@es-test /]#
Rallyの詳細については、公式ドキュメントをご参照ください：