Write operation in Elasticsearch ๐Ÿ’ฝ

Elasticsearch uses the write-ahead mechanism to maintain data integrity and reduce disk operation computation.

ยท

2 min read

Write operation in Elasticsearch ๐Ÿ’ฝ

Write operation is to insert new documents into the Elasticsearch. Changes are only written to disks during a Lucene Commit. Lucene commit is an expensive operation so can not be performed after each create or delete operation.

So let's see then how Elasticsearch performs write operation ๐Ÿ’ฅ

image.png

Write operations are not directly performed on disks but are recorded in Translog which is an in-memory record for create and delete operations.

  • Transactions are recorded in Translog
  • New documents are added to the memory buffer, which is unindexed in-memory storage.
  • A write operation is called successful only if it is recorded in primary translog and all replica translogs in other shards based on Quorum.

Refresh Operation

image.png

  • During this, the memory buffer content is copied to a newly created segment in the memory and translog is cleared.
  • And the newly created segment becomes available for search and indexing. -Refresh operation is executed in every second by default.

Flush Operation

image.png

This operation performs writing documents to disk. All new created in-memory segments and contents of memory buffer are merged to a new Lucene Segment and are committed to disk and translog gets empty. The main advantages of having a single merged segment are:

  • Discard older version documents and reduce index space on disk.
  • The old segments are removed and bigger segments are created which increases the search speed.

Conclusion

Disk write operations are expensive and Elasticsearch is designed to handle a large number of write requests in a production server with minimal latency and large throughput. Write operation in elasticsearch works on the write-ahead mechanism like in SQL databases. To know more about this topic please see ๐Ÿ‘‰ Reading and Writing Documents

ย