Add all streams in one collection

Following modifications are introduced in the MongoDB structure:

  • All streams for given data source are saved in one collection
  • new field message_id is used to save message id in data collection
  • inprocess_ collection also contains all streams. New fields message_id, stream and group_id are added
  • Indeses are created for data_, acks_ and inprocess_ collections
  • In order to keep back compatibility a message have to be modified before being returned:
    • copy message_id to _id
    • copy message_id to _id for messages of dataset
  • All acknowledged messages added to the same collection
    • New fields group_id and stream are added
Edited by Mikhail Karnevskiy

Merge request reports

Loading