[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: MongoDB Aggregation - Does $unwind order documents the same way as the nested array order? and tags: ['aggregation']

[User]
Text:I am wandering whether using $unwind operator in aggregation pipeline for document with nested array will return the deconstructed documents in the same order as the order of the items in the array.
Example:
Suppose I have the following documentsI would like to use paging for all values in all documents in my application code. So, my idea is to use mongo aggregation framework to:So the question using the example described above is:Is it guaranteed that the following aggregation pipeline:will always result to the following documents with exactly the same order?:
Code:{ "_id" : 1, "item" : "foo", values: [ "foo", "foo2", "foo3"] }
{ "_id" : 2, "item" : "bar", values: [ "bar", "bar2", "bar3"] }
{ "_id" : 3, "item" : "baz", values: [ "baz", "baz2", "baz3"] }
values[
    {$sort: {"_id": 1}},
    {$unwind: "$values"}
]
{ "_id" : 1, "item" : "foo", values: "foo" }
{ "_id" : 1, "item" : "foo", values: "foo2" }
{ "_id" : 1, "item" : "foo", values: "foo3" }
{ "_id" : 2, "item" : "bar", values: "bar" }
{ "_id" : 2, "item" : "bar", values: "bar2" }
{ "_id" : 2, "item" : "bar", values: "bar3" }
{ "_id" : 3, "item" : "baz", values: "baz" }
{ "_id" : 3, "item" : "baz", values: "baz2" }
{ "_id" : 3, "item" : "baz", values: "baz3" }

[Assistant]
Text:Hello @karaimin welcome to the community!There is a chance that you get the same results but no guarantee. Personally I always use the sort after the unwind, ideal with an $match before the unwind to try to get as few documents as possible.The above will provide for sure the wanted result. However, I hope that one oft the MDB internals can elaborate onCheers,
Michael
Code:$match[
    {$match: { <query> } }
    {$unwind: "$values"}
    {$sort: {"_id": 1}},
]

[User]
Text:Thank you @michael_hoeller for you response.
However, maybe this is not an option for me since I expect the real sample to have a small number documents which I will $match by some query  (this is why I am sorting them by _id) and large number of items in values array.
As I mentioned I want to use pagination for the items in the values attribute. So I need a consistent order of them. Sorting them is not an option since after $unwind I loose the DB Index and it will exceed the memory of 100MB available for the pipeline. I can use "allowDiskUse: true", but that is also not a good option because it will slow down queries a lot.
Code:valuesvalues"allowDiskUse: true"

[Assistant]
Text:Hello @karaiminI can use "allowDiskUse: true" , but that is also not a good option because it will slow down queries a lot.Yes, that should be avoid when you are acting in a realtime app.One thought on this, large arrays are almost always difficult to handle. Maybe you can change your schema and use embedding for $values? This would open completely different options.You can check if a multikey Index on one or some fields in your array can help. In case you can find a good index you can walk along the index without the $unwindsince after $unwind I loose the DB IndexI am not sure what you mean. When you deconstruct an array with $unwind you will get an document per field and element. You can add the option includeArrayIndex to $unwind. This will add an field to the new document which just numbers the fields. This could also provide a path to go.In general, to retrieve page ‘n’ the code looks like this:As the size of your data increases, this approach has performance problems.  The reason is that every time the query is executed, the full result set is built up, then the server has to walk from the beginning of the collection to the specified offset. As your offset increases, this process gets slower.  Also, this process does not make efficient use of the indexes.  So typically the ‘skip()’ and ‘limit()’ approach is useful when you have small data sets,The reason the previous approach does not scale well is the skip() command.  Depending how you have build your _id  you may can use the natural order in the stored data like a timestamp, or an index (pls s. above).This approach leverages the inherent order that exists in the “_id” field. Also, since the “_id” field is indexed by default, the performance of the find operation is very good.There are further options to do pagination:both I do not think will help here so I skip the examples.Cheers,
Michael
Code:"allowDiskUse: true"includeArrayIndex{ "_id" : 1, "arrayIndex" : NumberLong(0), "item" : "foo", values: "foo" }
{ "_id" : 1, "arrayIndex" : NumberLong(1),  "item" : "foo", values: "foo2" }
{ "_id" : 1, "arrayIndex" : NumberLong(2),  "item" : "foo", values: "foo3" }
{ "_id" : 2, "arrayIndex" : NumberLong(1),  "item" : "bar", values: "bar" }
{ "_id" : 2, "arrayIndex" : NumberLong(2),  "item" : "bar", values: "bar2" }
{ "_id" : 2, "arrayIndex" : NumberLong(3),  "item" : "bar", values: "bar3" }
// Page 1
db.users.find().limit (10)
// Page 2
db.users.find().skip(10).limit(10)
// Page 3
db.users.find().skip(20).limit(10)
db.users.find().skip(pagesize*(n-1)).limit(pagesize)
// Page 1
db.users.find().limit(pageSize);
// Find the id of the last document in this page
last_id = ...

// Page 2
users = db.users.find({` `'_id'` `> last_id}). limit(10);
// Update the last id with the id of the last document in this page
last_id = ...
$slice

[User]
Text:Hi @michael_hoellerI didn’t get your idea here:Maybe you can change your schema and use embedding for $values? This would open completely different options.If you mean changing my schema to looks similar to what $unwind does (reversing One-to-Many relation) and adding appropriate key index for each value will do the trick, but this schema also has a lot of concerns related to my other scenarios.I am not sure what you mean.I mean that now I may have a single key index on values  field (used for queries), but after $unwind it is useless. So, next stage (which is $sort) won’t benefit from the sorted structure of the indexes in the B-Tree just like regular $sort operation on indexed field.My example is similar to the bucket pattern. The large document (with a too large array) is partitioned to some small documents (with smaller array size) in order to fit in the max document size.
Keeping too many documents will lead to heavy update operations (sometimes including massive amount of documents ) which I try to avoid.
Code:values

[Assistant]
Text:The same question:Does $unwind keep the “unwound” documents in the same order as they were ordered in the source array?Need to know this as we would like to avoid an unneeded $sort that would require allowDiskUse: true (when the aggregation otherwise doesn’t). Couldn’t find anything about this in the docs.It seems potentially reasonable to assume that it would be in the same order, but can we be sure? @karaimin did you figure it out?
Code:$unwind$sortallowDiskUse: true

[User]
Text:No @qtax, I didn’t find appropriate solution. I didn’t find any information in the official documentation and according to @michael_hoeller the order is not guaranteed, so I assume that I don’t have any evidence to rely on the returned order.
I solve the problem with aggregation pipeline that sort the documents and give me the length of each nested array. After that with some application logic I am able to reduce the documents where my next page is. Then, I query only the filtered documents and this way the pagination is achieved. It is not so efficient like the assumption described in the first question but for now it is the best I have found.
Code:

[Assistant]
Text:Hello @karaimin and @qtax,thanks for updating, I lost your message on my bookmarks.according to @Michael the order is not guaranteedThis was my experience in an previous customer project. The “unwound” documents are  identical to the input document except for the value of the array field which now holds a value from the original  array element. It seems that the documents are created by the sequence they occur in the array. I assume that one unwind can be idempotent but within the the flow of a process you might get side effects (e.g. empty array).
In case you need to keep the sequence you can add includeArrayIndex which will add a field to the unwound document which holds the position of the field in the previous array.To be absolutely sure what is correct I hope that @Asya_Kamsky can step in?regards,
Michael
Code:arrayincludeArrayIndex

[User]
Text:Hi @michael_hoellerI know the existence of includeArrayIndex option. However I would like to avoid application level sorting. So, saying in other words:Is there any chance that this:may be destructed to this.Take a look at the order (1 0 2)Having only the index won’t solve our problem. We need application level sorting if the above sequence is returned.
Best Regards
Aleydin
Code:includeArrayIndex{ "_id" : 1, "item" : "foo", values: [ "foo", "foo2", "foo3"] }
{ "_id" : 1, "item" : "foo", values: "foo2", "arrayIndex" : NumberLong(1 }
{ "_id" : 1, "item" : "foo", values: "foo", "arrayIndex" : NumberLong(0 }
{ "_id" : 1, "item" : "foo", values: "foo3", "arrayIndex" : NumberLong(2 }

[Assistant]
Text:Hi @karaiminI understood your question, I have seen this happen in a project some month back. I did tests just for me, but I have not been able to recreate the issue. To be on the save side on customer projects, I did add an extra(?) sort.  This is a good question! With the last message I “pinged” @Asya_Kamsky she probably can add the full insight.
Just do not want to limt this to Asya she is often abroad, anyone else around who can add on this?Regards,
Michael
Code:

[User]
Text:Ok, Thank you @michael_hoeller for all your support. I am looking forward of further updates on this topic. I will be glad to see official statement from the MongoDB staff.
Code:

[Assistant]
Text:Hi there folks,Yes, the order of documents will be the same as the order of elements in the array and the order pre-unwind will be preserved.But using $skip and $limit for pagination is not really a good idea as it’s not very performant, especially once you move to sharding…Asya
Code:

[Assistant]
Text:Hello Asya,
thanks for clearing that, lucky to save an extra sort in the future. Concerning pagination I provided some suggestions  fairly at the beginning of the post.
Michael
Code:

[User]
Text:Thanks @Asya_Kamsky and @michael_hoeller for your time on revealing the mystery  …  and the suggestions about the pagination approach.Since, $unwind will be used, I won’t be able to benefit from an indexed field and $skip and $limit is my only option for pagination (with this data model). I am going to test it if it can suits our needs. Maybe, combination from both “$skip/$limit” for the first few pages and the approach I described above can be appropriate for me.Best Regards
Aleydin
Code:

[Assistant]
Text:Yes, the order of documents will be the same as the order of elements in the array and the order pre-unwind will be preserved.Thanks for sharing this statement, and I gather you are a very reputable source, having worked on the aggregation framework itself.It seems a colleague of yours contradicted this statement in a different forum thread a few months later sayng “MongoDB only gurantee order of documents based on a sort operation” which leaves me unsure whether it would be safe to avoid a sort operation (for performance reasons) when performing $unwindIs there any up-to-date definititve documentation on what guarantees there are regarding array ordering during aggregation?
Code:$unwind

[Assistant]
Text:There is no contradiction.The $unwind preserves the order of the documents it receives and the order of the array elements.To get a predictable order of documents you must $sort.That is if you $sort before $unwind, documents will be sorted after $unwind.
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Error in finding the user MongooseError: Operation `users.findOne()` buffering timed out after 10000ms and tags: ['node-js', 'mongoose-odm']

[User]
Text:i use cmd  in terminal  npm start    and it start at port 5000  and also connect mongo db  and when i brows localhost://5000 in browser, it open but when i sign up in it then in browser porcesssing only not any response.and when i see in terminal it  show me error  Express server is up and running on port 5000
Successfully connected to db cars
Error in finding the user MongooseError: Operation users.findOne() buffering timed out after 10000ms
at Timeout. (C:\Users\Irfan Ansari\Desktop\IssueTracker\node_modules\mongoose\lib\drivers\node-mongodb-native\collection.js:175:23)
at listOnTimeout (node:internal/timers:564:17)
at process.processTimers (node:internal/timers:507:7)
Error in finding the user MongooseError: Operation users.findOne() buffering timed out after 10000ms
at Timeout. (C:\Users\Irfan Ansari\Desktop\IssueTracker\node_modules\mongoose\lib\drivers\node-mongodb-native\collection.js:175:23)
at listOnTimeout (node:internal/timers:564:17)
at process.processTimers (node:internal/timers:507:7)so  pls solve it
Code:users.findOne()users.findOne()

[User]
Text:C:\Users\Irfan Ansari>mongosh
Current Mongosh Log ID: 63cbdffba0f2600a491b9ec3
Connecting to:          mongodb://127.0.0.1:27017/?directConnection=true&serverSelectionTimeoutMS=2000&appName=mongosh+1.6.2
MongoServerSelectionError: Server selection timed out after 2000 msC:\Users\Irfan Ansari>
Code:

[Assistant]
Text:you can try this: Go to data services tab security on the left side and find network access inside that you choose to add the IP address and add 0.0.0.0/0  this IP address means you can access to database from any network address
Code:

[Assistant]
Text:MongooseError: Operation users.findOne() buffering timed out after 10000ms
at Timeout. (C:\Users\Hassan Dawood\Documents\commercehope-backend\node_modules\mongoose\lib\drivers\node-mongodb-native\collection.js:
185:23)
at listOnTimeout (node:internal/timers:569:17)
at process.processTimers (node:internal/timers:512:7)
Code:users.findOne()


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: [MongoDB .local Events] MongoDB is heading out on a World Tour! and tags: ['london-mug', 'newyork-mug', 'saopaulo-mug', 'mexico-mug']

[User]
Text:Email_Local-Series1200×500 613 KBHello All!MongoDB is heading out on a world tour to bring the best, most relevant content directly to you! Join us to connect with MongoDB experts, meet fellow users building the next big thing, and be among the first to hear the latest announcements!Register Now with **MUG50** discount code to get 50% off! For India-based events use  Invite Code **MDB.Community.23** to register yourself for the event. Keynote Presentations: Learn about the latest product announcements from MongoDB industry leaders. Technical Sessions: Experience educational technical sessions for all levels, delivered by MongoDB experts and customers.Relational Migrator
Learn how to easily migrate a relational database to MongoDB with one-time or continuous data migration.Data Modeling
Learn data modeling principles, and take your models to the next level with tips and tricks from the experts.Atlas Search:
Explore the Atlas Search ecosystem and how you can use it to effortlessly integrate search into your applications. MongoDB Product Demos: Stop by the MongoDB booth for demos of the latest products and to get your questions answered. Networking: Meet with our MongoDB enthusiasts in your area to share ideas and expand your network.Join our regional virtual user groups to connect with MongoDB Community in your region and stay updated on events in your time zone!  Use code MUG50 for 50% off your ticket!!
 Use Invite Code MDB.Community.23 for .locals in IndiaUpcoming Events:We look forward to seeing you at these events!
Code:**MUG50****MDB.Community.23**

[Assistant]
Text:We’re looking forward to meeting community members at these events! Please share if you’re planning to attend!
Code:

[Assistant]
Text:Hey regarding the MongoDB.local Bangalore event, it’s mentioned that it’s free of cost to attend. In that case, should I use MUG50 as invite code?
Code:

[User]
Text:Hey @_karanel,
For India-based .locals you could use MDB.Community.23 to register for the event
Code:MDB.Community.23

[Assistant]
Text:Hey @Harshit, actually I registered with my work email (karan@phoenixrp.io) without a code entering it as “NA”, and I can’t register again.
Could you help me out?
Code:

[User]
Text:No problem @_karanel. You don’t need to register again.
We will let the team know about it Looking forward to seeing you at the event!
Code:

[Assistant]
Text:Hey @Harshit, I registered with my work email with a code entering it as “yogesh”,  I can’t register again.
Could you please help?
Code:

[User]
Text:Hey @yogesh_mishra,
Let us check with the team and help you out. Could you also in the meantime, use the “Contact the Organiser” button on the registration page to share your concern.https://events.mongodb.com/mongodb-local-delhincr
Code:

[Assistant]
Text:I am a student. Which Email should I use for .local Mumbai to register as it is not accepting my personal email. Should I use my college email instead?
Code:

[Assistant]
Text:Yes, you can probably use your college email.
Code:

[Assistant]
Text:Registered. See y’all there!
Code:

[Assistant]
Text:Thanks for sharing Harshit, would be there in the Boston Meetup.
Can you also update if there would be any events in Canada for this tour - Toronto/Vancouver?Thanks.
Code:

[Assistant]
Text:Hi Devang - thanks for your interest! We are starting a MongoDB User Group in Boston in late October after the .local event. You can go ahead and join here: Boston, us: mongodb user group - MongoDB Developer Community ForumsThere aren’t any .local events in Canada this year.
Code:

[Assistant]
Text:Robbed, Toronto was robbed I say!
Code:

[Assistant]
Text:Hi, I registered for the .local Bengaluru event, but I didn’t receive any confirmation email.Can you please tell by when we will receive the confirmation for this event?
Code:

[User]
Text:Hey @rakesh_verma24,
Our team is currently in the process of sending out confirmations for the event. We understand that this may cause some anticipation.  Also, preference is being given to professionals as the content is tailored to their needs.In the meantime, we invite you to join our Bangalore User Group to stay informed about any upcoming meetups in the region.Thanks
Harshit
Code:

[Assistant]
Text:26th August Ahmmedabad is missing.
Code:

[User]
Text:Hey, @Anjesh_Agrawal That is our MongoDB User Group Meetup event run and managed by the community leaders, while these .locals are larger day-long events, majorly run by MongoDB Staff. It has around 7-8 sessions, booths and ask the experts sections to interact with MongoDB staff.This year .locals in India are being held in Bengaluru, Delhi and Mumbai.
Code:

[Assistant]
Text:hello , i registered for MongoDB .local mumbai but didn’t received any acknowledgment about the event ,
i even wrote the mail to organizer didnt receive any response, either the computergenerated nor thereply mail
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: I am getting this error const serverSelectionError = new ServerSelectionError(); and tags: ['node-js', 'mongoose-odm']

[User]
Text:The application successfully started on 27017
D:\web\web developement\Dance website pug\node_modules\mongoose\lib\connection.js:807
Code:  const serverSelectionError = new ServerSelectionError();
                               ^

MongooseServerSelectionError: Invalid message size: 1347703880, max allowed: 67108864
    at NativeConnection.Connection.openUri (D:\web\web developement\Dance website pug\node_modules\mongoose\lib\connection.js:807:32)
    at D:\web\web developement\Dance website pug\node_modules\mongoose\lib\index.js:342:10
    at D:\web\web developement\Dance website pug\node_modules\mongoose\lib\helpers\promiseOrCallback.js:32:5
    at new Promise (<anonymous>)
    at promiseOrCallback (D:\web\web developement\Dance website pug\node_modules\mongoose\lib\helpers\promiseOrCallback.js:31:10)
    at Mongoose._promiseOrCallback (D:\web\web developement\Dance website pug\node_modules\mongoose\lib\index.js:1181:10)
    at Mongoose.connect (D:\web\web developement\Dance website pug\node_modules\mongoose\lib\index.js:341:20)
    at Object.<anonymous> (D:\web\web developement\Dance website pug\app.js:10:10)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10) {
  reason: TopologyDescription {
    type: 'Unknown',
    servers: Map(1) {
      '127.0.0.1:27017' => ServerDescription {
        _hostAddress: HostAddress { isIPv6: false, host: '127.0.0.1', port: 27017 },
        address: '127.0.0.1:27017',
        type: 'Unknown',
        hosts: [],
        passives: [],
        arbiters: [],
        tags: {},
        minWireVersion: 0,
        maxWireVersion: 0,
        roundTripTime: -1,
        lastUpdateTime: 2786731,
        lastWriteDate: 0,
        error: MongoParseError: Invalid message size: 1347703880, max allowed: 67108864
            at processIncomingData (D:\web\web developement\Dance website pug\node_modules\mongodb\lib\cmap\message_stream.js:91:18)
            at MessageStream._write (D:\web\web developement\Dance website pug\node_modules\mongodb\lib\cmap\message_stream.js:28:9)
            at writeOrBuffer (node:internal/streams/writable:389:12)
            at _write (node:internal/streams/writable:330:10)
            at MessageStream.Writable.write (node:internal/streams/writable:334:10)
            at Socket.ondata (node:internal/streams/readable:754:22)
            at Socket.emit (node:events:526:28)
            at addChunk (node:internal/streams/readable:315:12)
            at readableAddChunk (node:internal/streams/readable:289:9)
            at Socket.Readable.push (node:internal/streams/readable:228:10) {
          [Symbol(errorLabels)]: Set(0) {}
        }
      }
    },
    stale: false,
    compatible: true,
    heartbeatFrequencyMS: 10000,
    localThresholdMS: 15,
    logicalSessionTimeoutMinutes: undefined
  },
  code: undefined

[Assistant]
Text:did you get sollution to this bug?
Code:

[Assistant]
Text:This error is probably occur because the mongo dB services is not started yet what you have to do is
Go to control panel
Double click on services
then search for mongodb.exe and then click on start
its for window user
I hope it would be work.
Code:

[Assistant]
Text:/app/node_modules/mongoose/lib/connection.js:847
source_code-server-1   |   const serverSelectionError = new ServerSelectionError();
source_code-server-1   |                                ^
source_code-server-1   |
source_code-server-1   | MongooseServerSelectionError: connect ECONNREFUSED 127.0.0.1:27017After running the mongodb service i got the above error for docker
docker-compose.yamlversion: “3.8”services:mongodb:server:client:volumes:data:logs:dockerfileFROM node:18.8.0WORKDIR /appCOPY package*.json /app/RUN npm installCOPY . /app/EXPOSE 4000CMD [ “npm”, “start”]Please help me.
Code:image: "mongo"

volumes:

  - data:/data/db
build: ./server

ports:

  - "4000:4000"

volumes:

  - logs:/app/logs

  - ./server:/app

  - /app/node_modules

depends_on:

  - mongodb
build: ./client

ports:

  - "3000:3000"

environment:

  - CHOKIDAR_USEPOLLING=true

volumes:

  - ./client/src:/app/src

stdin_open: true

tty: true

depends_on:

  - server

[Assistant]
Text:Hey I think your problem is that you don’t have mongodb installed on your computer. Once you download it you can changeto
Code:mongodbmongoose.connect('mongodb://localhost:27017/blog')
mongoose.connect('mongodb://127.0.0.1/blog')

[Assistant]
Text:Thankyou this saved me haha
Code:

[Assistant]
Text:Ok, changing the URI worked but why?!?! Ok I’m using mongoose and found out why:from MS Edge
Code:

[Assistant]
Text:this worked for me. thank you
Code:

[Assistant]
Text:thank u, it was pain in my head
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Designing Mongo DB for Multi tenanted database and tags: ['data-modeling']

[User]
Text:We are currently in process of designing a Multi tenant application, we choose to go with Mongo db for data storage. Our application needs to store different entities, like user, employee, payroll etc., We want to build a scalable platform where clients can onboard/offboard faster, and manage the solutions easily. We also wanted to isolate the customer data logically.We are discussing few options on Mongo db how to place the data -Few queries -Many Thanks
Code:

[Assistant]
Text:Welcome to the community, @Thirumalai_M!Am I understanding correctly that the same application is to be used by multiple companies but with the same backend servicing all? Or will it be different deployments with a backend for each deployment? Generally you will want to store data for entities in collections so you’d want a users collection, an employees collection etc. Is the isolation of customer data so that you can give access to the data on a database level or are there other reasons? Will your clients be able to access the database directly or only via your application?So per your questions:I’m not sure what you mean with questions 3 and 4 - can you provide more information?
Code:{"name": "Naomi", "team": "Engineering Communications", ...}

[User]
Text:Thanks Naomi for the detailed response. Your answer to my queries gives what I feed for No1. I have few queries on No1 and will clarify for No3.Basically we are a SaaS provider company developing system, the tenants can be onboarded    and store their data. There are multiple entities in our application. We are using docker/container for service layer, but database wise we have some confusion. Whether separate database per client,  separate collection per client or documents to be segregated using partition key.As per your answer for No2, I understand going with one collection per client will create data model issue. Hence going with single database with one collection per client is not feasible approach.Using a one collection for one entity (for Ex: Employee) and having its own index/partition will not provide isolation. Our application is a heath care system, should be HIPPA compliance.I understanding going with a single database per client will make sense for our scenario. Please correct me if I am wrong in my understanding.
Code:

[Assistant]
Text:If you are using MongoDB Atlas, you can control access based on field values using Realm Rules. This would allow you to use one collection per entity while restricting access to the client that the data belongs to. So you could have a rule that a user can only see data that matches the company associated to that user.If you are not using Atlas you can control access on a database or collection level. So you could use that to restrict access for a database per client for your scenario.
Code:

[Assistant]
Text:@Thirumalai_M - my two cents - since it’s going to be a Healthcare/HIPPA compliant - better to go with  separate database per client, that gives the highest level of isolation and security and due to security & compliance: most clients usually may want to have the database deployed on their owned secured network (cloud or on-premise) only.  At the same time - it also allows doing client specific customization’s later, if required. Now, the major drawbacks of this approach is the cost and maintenance overhead etc.Had this been a non-healthcare system - you could have even thought of having single collection for all clients but segregating their rows/documents based on additional “tenant_id” column. Although, it only provides only logical level of isolation but it’s very easier to maintain and very cost effective. Depending on the number of clients/customers - you can easily scale it by having multiple shard clusters across -  may be 1 for each and/or group of customers combined into one cluster i.e. tenant_id = 1 to 10 or tenant_name = A to M and tenant_id = 11 to 20 or tenant_name = M to Z into another cluster.
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Advice for Chat schema design and tags: ['data-modeling']

[User]
Text:Hi, in my application I’m developing I need to allow for chats between any 2 users. I have the following schema in mind for a Messages collection:userID1 will always be the first user to start the chat, I’ll save this for every Chat room consisting of 2 people. (EDIT: Should I use lexicographical ordering instead, with userID1 coming before userID2?)I’ll create an index on the timestamp field so I can sort it in reverse, and then I can do the following to get the data every time the chat room is loaded (with pagination as you scroll up):And this I think should give me my intended behavior.Appreciate any advice, thanks.
Code:{
  _id: ObjectId(),
  userID1: "userA_ID",
  userID2: "userB_ID",
  sender: "one of the users",
  message: "some text",
  timestamp: time
}
userID1db.find({ userID1, userID2 }).sort({ "timestamp" : -1 })
  .skip(offset).limit(limit)

[Assistant]
Text:db.find({ userID1, userID2 }).sort({ "timestamp" : -1 })That particulate query will be better serve with a compound index on userID1,userID2,timestamp.  See Performance Best Practices: Indexing | MongoDB Blog for the specifics.
Code:db.find({ userID1, userID2 }).sort({ "timestamp" : -1 })

[Assistant]
Text:In your schema you might want also want to add a field “sent_by” or something similar, this way you could moderate / find messages by users with a search feature.
Code:

[User]
Text:Thanks for pointing this out, it’s a necessity actually. Otherwise I have no way of knowing who actually sent the message, oversight on my part.
Code:

[User]
Text:I wonder if I should use lexicographical sorting to determine userID1 and userID2, with userID1 coming before userID2, instead of who initiates the chat first.
Code:

[Assistant]
Text:Could you merge userID1, userID2 and sender into just 2 fields?sender rather than userID1
receiver rather than userID2This would reduce the size of each document.
Code:

[User]
Text:Thanks for the suggestion, that makes a lot of sense, then the accompanying query would be the following right?So I would then have to create a multikey index on the sender, receiver and timestamp fields?
Code:db.find({ $or: [{ sender: userA, receiver: userB }, { sender: userB, receiver: userA }]}
  .sort({ timestamp: -1 }).skip(offset).limit(limit)

[Assistant]
Text:Another implementation you could do that would involve another collection is, when a new chat is created in a “Conversation” collection you can add the information:Then in the message instead of UserID1 and UserID2 you would just have “conversation_id”This way if the members of the group change you only have to change it once in the “Conversation” collection and all messages referencing the ID will see the changes.This would be similar to a One-to-Many with reference
Code:
{ id: 123
  participants: ['user1', 'user2'],
}
{ sender: 'user1',
  message: 'Hello World',
  timestamp: time,
  converstationId: 123
}

[User]
Text:That’s a nice approach! Thank you both for the insights and advice.I was wondering (which was my second question), is storing a huge amount of random chat data in a collection good practice? I say random because in the Messages collection the ordering will be jumbled up when different users communicate at different times. Although it has no effect on the end users, is that acceptable practice?A more traditional data structure would be perhaps to keep all this chat in an array within a document (but of course it’s subject to the 16MB BSON size limit), but logically this means there’s no way any messages are interleaved with other messages.
Code:

[Assistant]
Text:In general I don’t believe that collections with a lot of documents is an issue.  As long as your queries are indexed it shouldn’t be a problem.Blockquote
A more traditional data structure would be perhaps to keep all this chat in an array within a document (but of course it’s subject to the 16MB BSON size limit), but logically this means there’s no way any messages are interleaved with other messages.MongoDB does have a bucket design pattern.  In which you store related items in an array.You could have a field called “total_messages” that is the sum of all messages and once it hits a certain number it creates a second bucket so you stay within the 16MB limit and don’t have the massive arrays anti pattern.
Although this may be more complicated than is required.
Code:{
    conversation_id: 12345,
    time: time,
    members: ['user1', 'user2'],
    messages: [
      {
         sender: 'user1',
         message: 'Hello World',
         timestamp: time
      },
      {
         sender: 'user1',
         message: 'Hello World',
         timestamp: time
      }],
   total_messages: 2
}

[User]
Text:Thank you for the links and explanation. That’s a nice approach as well, but yes it comes with a massive array regardless, and queries become a little more complex.What we’re doing now without using the bucket design pattern is essentially merging all of these potential arrays into one collection. I think I shall be going ahead with what’s been discussed so far with the sender/receiver method.
Code:

[Assistant]
Text:If you are working on a chat application that is great, I think you should have the feature to add more users.
Code:

[Assistant]
Text:Hi thanks for this… but i need to know how to get all the results by sortedI have used this… but thing is every things are in result block and I want it to in one array so I can fetch previous messages with $limit and $skip so I can send back to API of user message…
here is my sample dataI have made 3 messages collection document after 3 new document is created… i will make it 1000 in production…
and I have use another thing called $concatArray but its adding only fields which I explicitly give exampleit will add two result to messages  but not to all of that result
here my demo MongoDB collections [{ "_id": { "$oid": "63bd85a54ffc8dc686c0df5b" }, "message": [ - Pastebin.com
Code:
db.chats.aggregate(
    {
        /**
         * query: The query in MQL.
         */
        $match: {
          "members": {
            "$all": [
                ObjectId('63ad8631e0d6cfe452b80677'),
                ObjectId('63adadf287e482e6f128ec0e'),
            ]
          }
        },
    },
    {
        /**
         * Provide any number of field/order pairs.
         */
        $sort: {
          "created_at": -1
        }
    },
    {
        /**
         * specifications: The fields to
         *   include or exclude.
         */
        $project: {
            "_id": 0,
            // "members": 1,
            "result": {
                $sortArray: {input: "$message", sortBy: {created_at: -1}}
            }
        }
    },
)
result$limit$skip[
  {
    "result": [
      {
        "uuid": "df359053-5b69-4fc3-ac42-57aaf70ec9d9",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00011",
        "created_at": {
          "$date": "2023-01-10T19:52:26Z"
        }
      },
      {
        "uuid": "0cfe04ab-ccf2-4eb9-8760-d3471f033ea6",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00010",
        "created_at": {
          "$date": "2023-01-10T19:52:15Z"
        }
      }
    ]
  },
  {
    "result": [
      {
        "uuid": "fe54bf97-5dad-4d3f-ada8-ea1f1ab31039",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00010",
        "created_at": {
          "$date": "2023-01-10T19:52:14Z"
        }
      },
      {
        "uuid": "9face81d-b989-4662-a090-2f8f498b1a7b",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00010",
        "created_at": {
          "$date": "2023-01-10T19:52:12Z"
        }
      },
      {
        "uuid": "8578ce16-f3ac-4ceb-b779-0c1f3e8fec2f",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00010",
        "created_at": {
          "$date": "2023-01-10T19:51:02Z"
        }
      }
    ]
  },
  {
    "result": [
      {
        "uuid": "ef6f12e0-b601-4377-8ff9-28921799985b",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00009",
        "created_at": {
          "$date": "2023-01-10T19:51:00Z"
        }
      },
      {
        "uuid": "6fcbcc3b-ff09-4e1c-9ce5-d73790911592",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00008",
        "created_at": {
          "$date": "2023-01-10T19:50:47Z"
        }
      },
      {
        "uuid": "fdde26cd-6688-466a-be61-37ac8e312a70",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00007",
        "created_at": {
          "$date": "2023-01-10T19:49:33Z"
        }
      }
    ]
  },
  {
    "result": [
      {
        "uuid": "fdde26cd-6688-466a-be61-37ac8e312a70",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00007",
        "created_at": {
          "$date": "2023-01-10T19:49:33Z"
        }
      },
      {
        "uuid": "fb72bef2-ceff-4e22-80cd-b54da6ba6f01",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00006",
        "created_at": {
          "$date": "2023-01-10T19:48:51Z"
        }
      },
      {
        "uuid": "3ffee1e6-8eb4-4a93-99a2-329576dc7754",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00005",
        "created_at": {
          "$date": "2023-01-10T19:48:34Z"
        }
      },
      {
        "uuid": "927726bd-acc9-461b-98db-9ce07cca5ff0",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00004",
        "created_at": {
          "$date": "2023-01-10T19:48:31Z"
        }
      }
    ]
  },
  {
    "result": [
      {
        "uuid": "927726bd-acc9-461b-98db-9ce07cca5ff0",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00004",
        "created_at": {
          "$date": "2023-01-10T19:48:31Z"
        }
      },
      {
        "uuid": "664cb8fb-0284-4ad2-af1f-d9458fb09c24",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00003",
        "created_at": {
          "$date": "2023-01-10T19:48:25Z"
        }
      },
      {
        "uuid": "460e8a5a-327f-47c9-a8c3-76772cca4cc7",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00002",
        "created_at": {
          "$date": "2023-01-10T19:22:03Z"
        }
      },
      {
        "uuid": "ed4e5060-45f9-4f4a-a6a0-e45eddbd64bc",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00002",
        "created_at": {
          "$date": "2023-01-10T19:22:03Z"
        }
      }
    ]
  },
  {
    "result": [
      {
        "uuid": "a45e6738-afb4-47aa-a8f4-fdf1d70eeb5f",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00002",
        "created_at": {
          "$date": "2023-01-10T19:22:03Z"
        }
      },
      {
        "uuid": "466b8569-12c4-4d06-8572-eb5999f0e4ae",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00002",
        "created_at": {
          "$date": "2023-01-10T19:22:03Z"
        }
      },
      {
        "uuid": "314b60eb-1eaf-4a72-a7a6-528ab474a0d9",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00002",
        "created_at": {
          "$date": "2023-01-10T19:22:03Z"
        }
      },
      {
        "uuid": "460e8a5a-327f-47c9-a8c3-76772cca4cc7",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00002",
        "created_at": {
          "$date": "2023-01-10T19:22:03Z"
        }
      }
    ]
  },
  {
    "result": [
      {
        "uuid": "a45e6738-afb4-47aa-a8f4-fdf1d70eeb5f",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00002",
        "created_at": {
          "$date": "2023-01-10T19:22:03Z"
        }
      },
      {
        "uuid": "4f56f58d-1b0c-48f9-b3b6-c63020068760",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00002",
        "created_at": {
          "$date": "2023-01-10T19:21:55Z"
        }
      },
      {
        "uuid": "4fb86042-93ef-4209-88ca-8c287f16a2f6",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "00001",
        "created_at": {
          "$date": "2023-01-10T19:20:30Z"
        }
      },
      {
        "uuid": "3a6dc43d-bdb5-41b9-be93-15ab389380a9",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "New Docu ",
        "created_at": {
          "$date": "2023-01-10T18:44:01Z"
        }
      }
    ]
  },
  {
    "result": [
      {
        "uuid": "69b1dd00-8ecf-4ba3-8df4-2cbba3887505",
        "sender": {
          "$oid": "63adadf287e482e6f128ec0e"
        },
        "message": "88888888888888888888",
        "created_at": {
          "$date": "2023-01-10T16:01:22Z"
        }
      },
      {
        "uuid": "3ea03321-3c99-48b6-a7fd-2cc5fa514fb3",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "0000000000000",
        "created_at": {
          "$date": "2023-01-10T15:51:20Z"
        }
      },
      {
        "uuid": "c8e35012-1106-4e3f-b678-6040002d54b8",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "pppppp",
        "created_at": {
          "$date": "2023-01-10T15:40:24Z"
        }
      }
    ]
  },
  {
    "result": [
      {
        "uuid": "b78f1288-5363-4b0f-9bbe-ddf73f60fac1",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "Kio nn sass,,,",
        "created_at": {
          "$date": "2023-01-10T18:46:39Z"
        }
      },
      {
        "uuid": "94e3ae3a-eb2c-4ceb-b580-e97f4ec49aba",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "Kio nn sass,,,",
        "created_at": {
          "$date": "2023-01-10T18:44:57Z"
        }
      },
      {
        "uuid": "80c86174-9836-477b-b0d3-e4cb968b015a",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "Another messgee 2",
        "created_at": {
          "$date": "2023-01-10T18:44:45Z"
        }
      },
      {
        "uuid": "3a6dc43d-bdb5-41b9-be93-15ab389380a9",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "New Docu ",
        "created_at": {
          "$date": "2023-01-10T18:44:01Z"
        }
      },
      {
        "uuid": "69b1dd00-8ecf-4ba3-8df4-2cbba3887505",
        "sender": {
          "$oid": "63adadf287e482e6f128ec0e"
        },
        "message": "i am fine thanks",
        "created_at": {
          "$date": "2023-01-10T16:01:22Z"
        }
      },
      {
        "uuid": "3ea03321-3c99-48b6-a7fd-2cc5fa514fb3",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "Buddy hhh.. ",
        "created_at": {
          "$date": "2023-01-10T15:51:20Z"
        }
      },
      {
        "uuid": "c8e35012-1106-4e3f-b678-6040002d54b8",
        "sender": {
          "$oid": "63ad8631e0d6cfe452b80677"
        },
        "message": "LOL How are you ",
        "created_at": {
          "$date": "2023-01-10T15:36:24Z"
        }
      }
    ]
  }
]
$concatArray// MongoDB Playground
// Use Ctrl+Space inside a snippet or a string literal to trigger completions.

const database = 'test3';

// The current database to use.
use(database);

// db.chats.find({
//   "members": {
//     "$eq": [
//       ObjectId('63ad8631e0d6cfe452b80677'),
//       ObjectId('63adadf287e482e6f128ec0e')
//       ]
//     }
//   }
// )

// db.chats.aggregate(
// {
//   $match: {
//   "members": {
//     "$all": [
//       ObjectId('63ad8631e0d6cfe452b80677'),
//       // ObjectId('63adadf287e482e6f128ec0e'),
//       ObjectId('63b3c2b8ce51cf1fca0325ad')
//       ]
//     },
//   }
// },
// {
//   $sort: {
//     "created_at": -1
//   }
// },
// // {
// //   $project: {
// //     "total": 1,
// //     "_id": 0
// //   }
// // },

// {
//   $limit: 1
// },
// )
// .sort({"created_at": -1})

// db.chats.find(
//   {
//     "members": {
//       "$all": [
//       ObjectId('63ad8631e0d6cfe452b80677'),
//       ObjectId('63adadf287e482e6f128ec0e'),
//       // ObjectId('63b3c2b8ce51cf1fca0325ad')
//       ]
//     }
//   },
// )

// db.chats.find({
//   "members": {
//     "$eq": [
//       ObjectId('63ad8631e0d6cfe452b80677'),
//       ObjectId('63adadf287e482e6f128ec0e')
//       ]
//     }
//   }
// )

db.chats.aggregate(
    {
        /**
         * query: The query in MQL.
         */
        $match: {
          "members": {
            "$all": [
                ObjectId('63ad8631e0d6cfe452b80677'),
                ObjectId('63adadf287e482e6f128ec0e'),
            ]
          }
        },
    },
    {
        /**
         * Provide any number of field/order pairs.
         */
        $sort: {
          "created_at": -1
        }
    },
    {
        /**
         * specifications: The fields to
         *   include or exclude.
         */
        $project: {
            "_id": 0,
            // "members": 1,
            "result": {
                $sortArray: {input: "$message", sortBy: {created_at: -1}},
            }
        }
    },
    {
        /**
         * specifications: The fields to
         *   include or exclude.
         */
        $project: {
          "messages": {
            '$concatArrays': [
                "$result", "$result"
                ]
            }
        }
    }
)
resultmessagesresult

[Assistant]
Text:i solved this… but is it performance friendly
Code:
db.chats.aggregate(
    {
        /**
         * query: The query in MQL.
         */
        $match: {
          "members": {
            "$all": [
                ObjectId('63ad8631e0d6cfe452b80677'),
                ObjectId('63adadf287e482e6f128ec0e'),
            ]
          }
        },
    },
    {
        /**
         * Provide any number of field/order pairs.
         */
        $sort: {
          "created_at": -1
        }
    },
    // {
    //     /**
    //      * specifications: The fields to
    //      *   include or exclude.
    //      */
    //     $project: {
    //         "_id": 0,
    //         // "members": 1,
    //         "result": {
    //             $sortArray: {input: "$message", sortBy: {created_at: -1}},
    //             // "$initialValue": {},
    //         }
    //     }
    // },
    {
        /**
         * path: Path to the array field.
         * includeArrayIndex: Optional name for index.
         * preserveNullAndEmptyArrays: Optional
         *   toggle to unwind null and empty values.
         */
        $unwind: {
          path: "$messages",
        //   includeArrayIndex: 'string',
        //   preserveNullAndEmptyArrays: boolean
        }
    },
    {
        /**
         * specifications: The fields to
         *   include or exclude.
         */
        $project: {
            "_id": 0,
            "messages": 1
            // "members": 1,
            // "result": {
            //     $sortArray: {input: "$message", sortBy: {created_at: -1}}
            // }
        }
    },
    {
        /**
         * Provide any number of field/order pairs.
         */
        $sort: {
          "messages.created_at": -1
        }
    },
    {
        /**
         * outputFieldN: The first output field.
         * stageN: The first aggregation stage.
         */
        $facet: {
          data: [
                {$skip: 0},
                {$limit: 2}
            ],
            pagination: [
                {$count: "count"}
            ]
        }
    }
    // {
    //     /**
    //      * Provide any number of field/order pairs.
    //      */
    //     $sort: {
    //       "created_at": 1
    //     }
    // }
    // {
    //     $unwind: '$message',
    // }

    // {
    //     /**
    //      * specifications: The fields to
    //      *   include or exclude.
    //      */
    //     $project: {
    //         "result": {
    //             $sortArray: {input: "$message", sortBy: {created_at: -1}}
    //         }

    //     }
    // }
)

[Assistant]
Text:if you allow me then I’ll be the first beta tester of your chat schema design, which I used on my website page.
Code:

[Assistant]
Text:Of course, you can use it. and if you can please improve it. thank you
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Change dbPath on Ubuntu and tags: ['containers']

[User]
Text:Hello,I want to customise (change) the dbPath on ubuntu 20.04.The configuration file:/etc/mongod.confspecifies the data directory/dbPath:/var/lib/mongodbThe docs here: https://docs.mongodb.com/manual/reference/configuration-options/#storage.dbPathnote that on linux one can’t simply change the dbPath in the config file:" The Linux package init scripts do not expect  storage.dbPath  to change from the defaults. If you use the Linux packages and change  storage.dbPath , you will have to use your own init scripts and disable the built-in scripts."Unfortunately, I haven’t found any information on those init script(s), i.e. where I can find them, and what in them might need editing in order for the change of dbPath in the config. file to be allowed to work.Because I’m using ubuntu the init system is systemctl, see here: https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/Aside: why do I want to change the dbPath from its default? Because my laptop is dual boot and I have mongodb on both win10 and unbuntu, but want to be able to access the same database files–i.e. access the win10 directory/data storage from within the mongodb on ubuntu. So I think I want to use something like:/media/[username]/[win10 partition name]/data/dbwhich corresponds to:C:\data\dbBut as the docs say, the init script(s) also need changing for that to work.Thanks for your help.
Code:storage.dbPathstorage.dbPath

[Assistant]
Text:Changing the init script, if running systemd, is not that complicated.The commandsudo systemctl status mongodwill tell you the location of the script.  On my system, it is /usr/lib/systemd/system/mongod.service.  You will see in this text file some ExecStartPre= statement.  You just have to match those directories and files to the one specified in your mongod.conf.  You will see that these are probably only needed at the first start and it is too make sure that the directories exist and are writable by mongod user.But that’s the theory.  On a production server I do not mess with this.  On a dev machine I start mongod manually with the config file I pleased.  Sometimes I have different config files that point to different directories depending of the project.  Just make sure directories exist and are writable by the user that starts mongod.
Code:sudo systemctl status mongod

[Assistant]
Text:On top of what @steevej mentioned, you will also need to set the correct permissions on the directory. MongoDB in linux will need your new dbPath location to be writable by the mongod user. When you install MongoDB in Linux, the package will create the mongod user and group and then the /data/db path will be owned by this user/group. On a dual boot system, I don’t know what will happen if you try to change the ownership of a path on a Windows mount. You might need to open the path up to be world read/writable (chmod 777 /media/[username]/[win10 partition name]/data/db).
Code:dbPathmongodmongod/data/dbchmod 777 /media/[username]/[win10 partition name]/data/db

[User]
Text:Thanks. Here’s the result when I tried that:When I look inside the /lib/systemd/system/mongod.service file I see:It may just be my lack of knowledge about linux but I don’t see the sort of statement your describe. There is the line:ExecStart=/usr/bin/mongod --config /etc/mongod.confbut that’s the same config file I earlier wrote of wanting to be able to change–but when I do so, mongod will not start.Is this line important?:EnvironmentFile=-/etc/default/mongodThe thing is, I don’t know what the leading - is doing there. There’s no file at:/etc/default/mongod
Code:user@machine:~$ sudo systemctl status mongod
[sudo] password for user:
● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset>
     Active: active (running) since Wed 2020-06-03 18:28:28 NZST; 5h 5min ago
       Docs: https://docs.mongodb.org/manual
   Main PID: 902 (mongod)
     Memory: 209.8M
     CGroup: /system.slice/mongod.service
             └─902 /usr/bin/mongod --config /etc/mongod.conf

Jun 03 18:28:28 machine systemd[1]: Started MongoDB Database Server.
[Unit]
Description=MongoDB Database Server
Documentation=https://docs.mongodb.org/manual
After=network.target

[Service]
User=mongodb
Group=mongodb
EnvironmentFile=-/etc/default/mongod
ExecStart=/usr/bin/mongod --config /etc/mongod.conf
PIDFile=/var/run/mongodb/mongod.pid
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# locked memory
LimitMEMLOCK=infinity
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false

# Recommended limits for for mongod as specified in
# http://docs.mongodb.org/manual/reference/ulimit/#recommended-settings

[Install]
WantedBy=multi-user.target

[Assistant]
Text:Is this line important?:EnvironmentFile=-/etc/default/mongodThe thing is, I don’t know what the leading - is doing there. There’s no file at:/etc/default/mongodThe EnvironmentFile bit states that the service should look in the provided file for environment variables. The - in front of the file means to not error out if the file doesn’t exist. You can read more about EnvironmentFile in the systemd man page.The first place that I would start is changing the the path in the MongoDB config file. You can find the location of that file by looking at the ExecStart line:ExecStart=/usr/bin/mongod --config /etc/mongod.confIn your case, the file is located at /etc/mongod.conf. Edit that file and change the strorage.dbPath location to the path you want. Save and exit from the file. After doing that run sudo systemctl restart mongod, check the status of the service with systemctl status mongod and verify if it’s running. If it is you’re all set. If it’s not you need to check the MongoDB log file (default should be /var/log/mongodb/mongodb.log I believe) to see why the service failed.
Code:EnvironmentFile-ExecStart/etc/mongod.confstrorage.dbPathsudo systemctl restart mongodsystemctl status mongod/var/log/mongodb/mongodb.log

[User]
Text:Good point by both you (Doug) and Steeve, earlier. These are the current permissions for that dir:user@machine:~$ ls -l /media/[user name]/[win10 partition name]/data
total 8
drwxrwxrwx 1 owner group 8192 Jun  3 15:43 dbSo I think that shows access permissions are not what’s preventing mongod working if I change the config file’s dbPath to that target dir.?Would seeing the status help? After changing the dbPath in the config file to my target dir when mongod is stopped, I then:NB, '‘user’ and ‘machine’ are my placeholders.
Code:user@machine~$ sudo service mongod start
user@machine:~$ sudo service mongod status
● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor pres>
     Active: failed (Result: exit-code) since Thu 2020-06-04 01:37:16 NZST; 3>
       Docs: https://docs.mongodb.org/manual
    Process: 33255 ExecStart=/usr/bin/mongod --config /etc/mongod.conf (code=>
   Main PID: 33255 (code=exited, status=100)

Jun 04 01:37:16 machine systemd[1]: Started MongoDB Database Server.
Jun 04 01:37:16 machine systemd[1]: mongod.service: Main process exited, c>
Jun 04 01:37:16 machine systemd[1]: mongod.service: Failed with result 'ex>

[Assistant]
Text:The next step is to look at the the log file to see why it the mongod process failed. The service status just states that the process failed, but the log file will give more information about why.
Code:mongod

[User]
Text:OK, thanks. Here’s nearly the last bit from that log file:2020-06-04T01:37:16.571+1200 I  CONTROL  [initandlisten] options: { config: “/etc/mongod.conf”, net: { bindIp: “127.0.0.1”, port: 27017 }, processManagement: { timeZoneInfo: “/usr/share/zoneinfo” }, storage: { dbPath: “/media/[user name]/[win10 partition name]/data/db”, journal: { enabled: true } }, systemLog: { destination: “file”, logAppend: true, path: “/var/log/mongodb/mongod.log” } }
2020-06-04T01:37:16.572+1200 I  STORAGE  [initandlisten] exception in initAndListen: Location28596: Unable to determine status of lock file in the data directory /media/[user name]/[win10 partition name]/data/db: boost::filesystem::status: Permission denied: “/media/[user name]/[win10 partition name]/data/db/mongod.lock”, terminatingI don’t think it’s simply permissions on that file itself, though, as:user@machine:/media/[user name]/[win10 partition name]/data/db$ ls -l mongod.lock
-rwxrwxrwx 2 owner group 0 Jun  3 15:43 mongod.lockIs the file ownership itself important? The owner is not mongodb but rather my user name, while mongodb is the owner for the corresponding file on my linux partition’s default dbPath, i.e.:
/var/lib/mongodb/mongod.lockuser@machine:/var/lib/mongodb$ ls -l mongod.lock
-rw------- 1 mongodb mongodb 0 Jun  4 01:35 mongod.lockI tried changing the owner & group to mongodb but after that mongod still failed to start, with the log:2020-06-04T02:16:26.635+1200 I  CONTROL  [initandlisten] options: { config: “/etc/mongod.conf”, net: { bindIp: “127.0.0.1”, port: 27017 }, processManagement: { timeZoneInfo: “/usr/share/zoneinfo” }, storage: { dbPath: “/media/[user name]/[win10 partition name]/data/db”, journal: { enabled: true } }, systemLog: { destination: “file”, logAppend: true, path: “/var/log/mongodb/mongod.log” } }
2020-06-04T02:16:26.637+1200 I  STORAGE  [initandlisten] exception in initAndListen: Location28596: Unable to determine status of lock file in the data directory /media/[user name]/[win10 partition name]/data/db: boost::filesystem::status: Permission denied: “/media/[user name]/[win10 partition name]/data/db/mongod.lock”, terminatingedit: correction: my attempt to change the owner & group of the mongod.lock file in the target win10 partition actually failed, but I’m still not sure whether file ownership is important given everyone has rw access?
Code:

[Assistant]
Text:What are the permissions on the /media/[user]/[win10 partition]/data/db folder? My guess is they are wide open as well, but worth checking.MongoDB is trying to access the lock file but it can’t because the permissions of the file are rw by the owner only. You could try removing the mongod.lock file to see if you can start the service in Linux. I am not sure how that would affect things when you go back to your Windows version however.I get what you’re trying to do having MongoDB available no matter whether you boot into Windows or Linux, but I’m not sure that’s going to work well. If you have Windows 10 and enabled WSL this might work a little more smoothly as you’d have the same permissions and owners between the two systems. Trying to get the permissions on the files correctly and in a way that doesn’t affect the other OS might be more hassle than it’s worth.A couple of options that might be a better choice:Unfortunately I don’t have a dual boot Windows machine so I’m not able to test things out to see what would be needed to make this work on your set up.
Code:/media/[user]/[win10 partition]/data/dbrwmongod.lock

[Assistant]
Text:That would be the way to go.A couple of options that might be a better choice:The free tier is very nice for that sort of things because it is accessible everywhere.  Since I switch from a laptop on the go and a desktop at home having my data no matter what.
Code:

[Assistant]
Text:Boa Notie. ! Td BEm !
Estou enfrentando o mesmo problema. Porem utilizamos o Oracle Linxu.
Criamos um novo diretorio e alteramos o arquivode configuração para apontar para o novo caminhio.Revisamos todas as permissões do diretorio porem o erro acontece. Sabem o que poder.Unable to determine status of lock file in the data directory /media/[user name]/[win10 partition name]/data/db: boost::filesystem::status: Permission denied: “/media/[user name]/[win10 partition name]/data/db/mongod.lock”, terminatingI don’t think it’s simply permissions on that
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Djongo NotImplementedError: Database objects do not implement truth value testing or bool() and tags: ['python']

[User]
Text:Python 3.10.2
Django==4.1.1
PyMongo==4.2.0
djongo==1.3.6
while running migrate
NotImplementedError: Database objects do not implement truth value testing or bool(). Please compare with None instead: database is not None
Reinstall pymongo pymongo==3.12.3 resolved this problem.
Code:

[User]
Text:I’m using mongodb 6.0.1 Community,maybe should be wating for next version of djongo
Code:

[Assistant]
Text:Welcome to the MongoDB Community @John_Wang !The Djongo open source project currently isn’t very actively maintained, so I suspect you’ll be waiting a long while for bug fixes. You might be able to find more advice via the community resources in the Djongo Readme: Djongo Questions and Discussions.Reinstall pymongo pymongo==3.12.3 resolved this problem.Djongo 1.3.6 was released in June 2021 almost six months before the PyMongo 4.0 driver, so presumably needs some updates for full compatibility. Thanks for sharing your workaround.Can you share more details on how you are trying to use MongoDB with Django? For example: are you trying to run everything (including Django admin UI) with MongoDB as a backend database?I would personally be inclined to use an official driver like PyMongo or Motor with one of the Python microframeworks. The Django core is designed around SQL and isn’t a great fit for MongoDB at the moment. Alternatively, you could use a driver like PyMongo in your Django views but would still need a supported relational database for core Django admin features.Regards,
Stennie
Code:

[Assistant]
Text:To address the original error:NotImplementedError: Database objects do not implement truth value testing or bool(). Please compare with None instead: database is not NoneThis was an intentional minor breaking change we made in PyMongo 4.0 described here:Database.__bool__ raises NotImplementedError¶
Database now raises an error upon evaluating as a Boolean. Code like this:if database:
Can be changed to this:if database is not None:
You must now explicitly compare with None.https://pymongo.readthedocs.io/en/stable/migrate-to-pymongo4.html#database-bool-raises-notimplementederrorThe full trackback of the exception will show which line needs to be changed.
Code:Database.__bool__if database:if database is not None:

[Assistant]
Text:it’s resolved
after
pip install pymongo==3.12.3
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Can't create a unique index that ignores nulls in MongoDB and tags: ['data-modeling', 'java']

[User]
Text:MongoDB’s semantics of null is not the same as the semantics of null in SQL databases. As a consequent, unique indexes in MongoDB will raise uniqueness violation error when more than one document has null for the field that has unique index. Furthermore, there appears to be no way to create a unique index where nulls are ignored.MongoDB seems to have adopted the null value semantics of Java where null is a literal value assigned to variables of reference type to indicate that it has no referent. Thus, in Java null == null is true.However in the database world, null could have many meanings including indicating that the value of an attribute is unknown. Since two unknowns cannot be compared, null == null should be null, i.e., it is unknown if two unknowns are the same. With this semantics, two null values in a unique index are never equal in that null==null is not true. Thus unique indexes will allow multiple null values.This is an issue in migrating SQL database to MongoDB.
Code:

[Assistant]
Text:Hi @Rafiul_Ahad ,You should have a look at sparse indexes and, for even more flexibility, partial indexes.Regards,
Jeff
Code:

[User]
Text:I did. And nothing worked so far.
Code:

[User]
Text:From indexing - Unique index in mongoDB 3.2 ignoring null values - Stack Overflow, the following works for unique index for field of type String with nulls
IndexOptions indexOptions = new IndexOptions().unique(true).partialFilterExpression(Filters.eq(“fieldName”,Filters.eq(“$type”,“string”)));
but the note says:
"Just a note from my testing for anyone else looking for the same solution: although this worked as a mechanism for enforcing uniqueness while allowing nulls, Mongo didn’t use the index for lookup queries using the indexed field and using hint() to force it resulted in very slow performance– Pete S "Apr 20, 2018 at 10:25
Code:

[Assistant]
Text:Welcome to the MongoDB Community @Rafiul_Ahad!MongoDB’s semantics of null is not the same as the semantics of null in SQL databases.Null values are a placeholder for unknown values in both cases, but semantics for comparison and indexing may differ depending on the databases you are comparing. For example, SQL Server used to (and may still) disallow multiple nulls in a unique index by default – you have to create a filtered unique index (analogous to a partial index in MongoDB).I think the most straightforward way to ignore null values for the purposes of a unique index would be to avoid serialising null property values. For example, if you are using the MongoDB Java driver’s POJO support this happens by default (see Serialization Customization).For more information on null semantics in MongoDB, please refer to Query for Null or Missing Fields.I did. And nothing worked so far.It seems like you may have found an acceptable solution for your use case.If that isn’t the case or you are looking for further advice on this topic, please provide some more information on your environment:Per the answer you referenced from Stack Overflow, you can also use a partialFilterExpression matching on type to index a subset of documents based on field type (eg those where the fieldName value is a string ). However, as mentioned in the Partial Index Query Coverage documentation a partial index filter will not be chosen by the query planner unless a query includes the same filters:MongoDB will not use the partial index for a query or sort operation if using the index results in an incomplete result set.To use the partial index, a query must contain the filter expression (or a modified filter expression that specifies a subset of the filter expression) as part of its query condition.Regards.
Stennie
Code:nullnullnullnullpartialFilterExpressionfieldNamestring

[User]
Text:Hi Stennie,
Thank your for your response.
My current workaround is just that, a workaround to be able to create unique indexes on fields that have nulls in them. Since it comes with a performance penalty, it is not acceptable.
Let me get on the soapbox for a moment. Indexes are physical database constructs. They should never be seen by the application. Asking an application to include the filter used in the index in their query is not acceptable.
Without divulging much about what I am doing, let’s say that an application is programmatically generating data for insertion into MongoDB or MySQL database. It is also translating queries from a high level language to database query language. For MongoDB, this application will have to deal with this MongoDB idiosyncrasy adding to the complexity of the code.
To answer your questions, I am using MongoDB server version 6.0.0 and the application is using MongoDB Java Sync driver version 4.7.1. Initially the unique indexes were created using the following option:
IndexOptions indexOptions = new IndexOptions().unique(true);
But that caused uniqueness violation exception so I included the partialFilterExpression that I mentioned in this thread.
Thanks.
-Rafiul
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Update a collection field based on another collection and tags: ['aggregation']

[User]
Text:Hi,I need to update a collection field based on the values from another collection, For eg:
collection 1 - employee
values
{"_id":1,“name”:“aaa”, “department”:11}
{"_id":2,“name”:“bbb”, “department”:12}collection 2 - department
{"_id":11,“name”:“dept1”}
{"_id":12,“name”:“dept2”I need the result collection as
values
{"_id":1,“name”:“aaa”, “department”:{"_id":11,“name”:“dept1”}}
{"_id":2,“name”:“bbb”, “department”:{"_id":12,“name”:“dept2”}}Please let me know how can I update the existing column or create new column in the collection to get the result like the aboveThanks,
Code:

[Assistant]
Text:To that sort of things, you use the aggregation framework.  You can read about it https://docs.mongodb.com/manual/aggregation/.  MongoDB University also offer the M121 course related to it.You will need a $lookup aggregation stage.  Find more at https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/It is not clear if you want to add the field to the employee collection or if you want a third collection.  If the latter, you can achieve that with $out stage.  More at https://docs.mongodb.com/manual/reference/operator/aggregation/out/I recommend to go via $out even if the goal is to add the field in the first collection.  This way you do not screw up existing data and it gives you the opportunity to verify the results and simply swap the collection when you are happy.I also recommend using Compass to develop the pipeline as it makes it easy to experiment.  You can export the pipeline to your favorite programming language once you are done. This being said, I prefer to keep my pipeline in their natural json form in a file that I can reuse in the shell or in my favorite language.
Code:

[Assistant]
Text:You can perform a join, but a join is based a particular equality or an expression. If _id in the second table, is 10 + _id in the first table, then the join might be possible.However, the piped value is returned as an array, therefore, the array needs to be ‘unwinded’ and then manipulated into the document you want.Finally, you need to push this data into a new collection, which is quite easy once you actually get the data.
Code:

[Assistant]
Text:You can perform a join, but a join is based a particular equality or an expression. If _id in the second table, is 10 + _id in the first table, then the join might be possible.It is $lookup in MongoDB.  Join are SQL.  I really do not understand what you mean by if _id is 10+ in the first table.  Could you please elaborate?However, the piped value is returned as an array, therefore, the array needs to be ‘unwinded’ and then manipulated into the document you want.The exact stage is called $unwind as documented in https://docs.mongodb.com/manual/reference/operator/aggregation/unwind/and then manipulated into the document you want.Quite right, manipulated with $project and/or $addFields.And finallyFinally, you need to push this data into a new collection, which is quite easy once you actually get the data.That’s the $out stage already mentioned.
Code:

[Assistant]
Text:$lookup is the operator to perform a join, you’re right.I really do not understand what you mean by if _id is 10+ in the first table. Could you please elaborate?A person wants to join the documents, but a person needs some sort of expression to actually compare one document with another. I thought the expression is -_id(document in coll 1) = 10 + _id(document in coll 2)
Code:

[Assistant]
Text:The _id in collection 1 is not related at all to the _id of collection 2.What’s related is the field “department” in collection 1 and _id of collection 2.
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: CORS issue with client side Realm GraphQL Endpoint and tags: ['graphql']

[User]
Text:Hi everyone,First off - Kudos to the Realm team for building a great product. I am having a CORS issue when working with the GraphQL endpoint from the browser and hoping someone can lead me in the right direction. My issue is this: (apologies for the somewhat long explanation but I wanted to be as detailed as I could)I am using a custom JWT Authentication solution, so I currently maintain all tokens for identity etc already in my app. I am able to setup a  Custom JWT Authentication Authentication provider and input the corresponding JWK URI successfully in the Realm dashboard. It works fine after testing.The problem I have is then trying to pass in the JWT in the header of the post request to Realm GraphQL endpoint https://realm.mongodb.com/api/client/v2.0/app//graphql from the browser using fetch. When I do so I get the below CORS error:Access to fetch at ‘https://realm.mongodb.com/api/client/v2.0/app//graphql’ from origin ‘http://localhost:3000’ has been blocked by CORS policy: Response to preflight request doesn’t pass access control check: No ‘Access-Control-Allow-Origin’ header is present on the requested resource. If an opaque response serves your needs, set the request’s mode to ‘no-cors’ to fetch the resource with CORS disabled.Please note that I am not currently using the realm-web package as I do not want to add an additional JWT token layer  (realm-web maintains their own JWT token system in local storage) to my app that already has its own. I simply want to include the JWT token in the header of a fetch request to then have Realm verify against the JWK URI setup in the Realm dashboard. The docs here Authenticate GraphQL Requests make it seem like this is entirely possible as it notes that you can pass the custom JWT trough as a jwtTokenString header using a normal http request. However, I still get the CORS error when attempting this request.I have also tried whitelisting my http://localhost:3000 in the Realm dashboard under Manage > Settings > Allowed Request Origins. Still not able to get it to work.I was able to get several less than ideal workarounds working:Using another server as a proxy - since a node.js environment doesn’t have these CORS restrictions I was able to pass the token through to another endpoint which fetched the Realm GraphQL endpoint from a server environment successfully.  (Not great as that essentially creates 2 servers)I was able to use both the realm-web package and my own system to make a more complicated auth system  in which the realm-web package gave me a Bearer token to use without any CORS issues. (Not ideal as that requires 2 token systems)Potential Solutions:
I’m not positive on why the error is happening but I believe it may have something to do with the following that someone on the Realm team would have more context on:Are the URLS whitelisted in Allowed Request Origins mapped to the approved URLS for CORS requests?
Is the Access-Control-Allow-Headers option on the wherever the Realm server is hosted allowing a header of jwtTokenString ?Really appreciate anyone who can help lead me in the right direction on this.Thank you!
Code:jwtTokenStringjwtTokenString

[Assistant]
Text:Hi @Matt_Cunningham,Are you certain you use a PSOT method to run a qraphql query?Using an HTTPs API to run the query should be done via a POST method as defined here:Also you can use the Bearer method with an Access token you get from your JWT provider.I have noticed those CORS issues when trying to use other HTTP methods like “GET”.Best regards,
Pavel
Code:

[User]
Text:Hi @Pavel_Duchovny thanks for the reply, yes I am using a POST request and I am confident I am running the right query because it works in a server environment. When I switch to the browser with the same query it gives CORS issues. I am using something like this, passing in the JWT as jwtTokenString shown here Custom JWTAnother interesting note, I found these docs Authenticate HTTP Client Requests which statesMongoDB Realm enforces non-sync rules and sync rules for all client operations. This means that requests must be made by a logged in user of your Realm app.And those docs also offer the ability to get a Client API Access Token through the login endpoint.But I am curious what that means - does that mean I must use the realm-web package or ping the Realm login endpoint to receive a Realm generated Bearer token?In my case I am still wondering if I am able to just provide my custom JWT along with all of my requests to the endpoint and not have to generate another Bearer token through the login endpoint or realm-web package. Does that make sense?
Code:jwtTokenString    const result = await fetch(
      `https://myrealmgraphqlendpoint`,
      {
        method: "POST",
        headers: {
          jwtTokenString: user.token,
        },
        body: JSON.stringify({ query: FIND_MOVIES }),
      }
    );
    const json = await result.json();
    console.log(json);

[Assistant]
Text:Hi @Matt_Cunningham,I think it should work both ways otherwise it might be a bug.You should be able to provide credentials to do both auth + query. It can be email/password but also jwt token.Out of curiosity does a bearer token with access token doesn’t yield cors errors?Best
Pavel
Code:

[User]
Text:@Pavel_Duchovny Yeah I think it may be a bug. So I tried using Bearer with my JWT from my provider (not one assigned from Realm) like so:But now I get the below error in the console, which is probably because the Bearer strategy on the Realm endpoint is expecting a Realm assigned token format, not custom JWT format which maybe is why the jwtTokenString header exists in the first place.If it is a bug, I am happy to write out any further steps to reproduce. Let me know!
Code:  const result = await fetch(
      `https://myrealmgraphqlendpoint`,
      {
        method: "POST",
        headers: {
          Authorization: "Bearer " + user.token,
        },
        body: JSON.stringify({ query: FIND_MOVIES }),
      }
    );
jwtTokenString{
  "error": "value of 'kid' has invalid format",
  "link": "https://realm.mongodb.com/groups/5f3ab9628951c83aa903a0b0/apps/5f5d28bcdda1ce73d48eaa42/logs?co_id=5f5f9cb6a93317dab797e984"
}

[Assistant]
Text:@Matt_Cunningham,The bearer token can’t be your custom one but only realms.I asked  you to test it in 2 steps get a realm access token from custom-jwt/login endpoint and use it in the bearer.Pavel
Code:

[User]
Text:Hi @Pavel_Duchovny,The 2 steps process works when I use the login endpoint -  the login endpoint responds with an access_token and refresh_token etc. Using the provided access_token I am able to to query using the Bearer method successfully. While that process works and gives no CORS errors, it would mean I have to deal with handling refreshes of my own custom JWTs as well as the access_token that is provided back in the payload from the Realm login endpoint. Sorry if I am being repetitive here, but as you mentioned in your earlier replyYou should be able to provide credentials to do both auth + query. It can be email/password but also jwt token.Which means there should be a scenario where I wouldn’t have to use the login endpoint at all correct?
Code:

[Assistant]
Text:Hi @Pavel_Duchovny,I’m facing the same problem. I’m using an apiKey header and I’m getting the same error @Matt_Cunningham is getting.The thing is when I try the endpoint using Postman it works but when I use Apollo client with Angular it does not!!I hope that you have any idea about whats happening.
Code:

[Assistant]
Text:Hey Hadi -I’m assuming you’re running into this error because you’re calling it from the browser.  Please consider using authorization headers https://docs.mongodb.com/realm/graphql/authenticate/#credential-headers
Code:

[Assistant]
Text:me too, having the same error
from browser
with apiKey or authorization, getting CORS error
i added on App Settings > Allowed request origin: localhost but not worked too
Code:

[Assistant]
Text:HiIs there an update for solution on this?  Perhaps an example of how to update the apiKey header?Many thanks!
Code:

[Assistant]
Text:The API key shouldn’t be used directly in the browser. Please authenticate with the API token like so.and then use the access token to authenticate to GQL as Pavel mentioned
Code:

[Assistant]
Text:I’m unable to get past CORS error (Access to fetch at … from origin … has been blocked by CORS policy: Response to preflight request doesn’t pass access control check: No ‘Access-Control-Allow-Origin’ header is present…) using the browser to query the GQL endpoint.  I am passing the bearer token issued by the login endpoint and the POST request works fine in Postman, including receiving the required ‘Access-Control-Allow-Origin’.However, in the browser, the request first sends the OPTIONS request to verify CORS, and the response from the server does not contain the requisite “Access-Control-Allow-Origin”.  I can duplicate this result in Postman as well by setting the request type to OPTIONS and including all of the necessary header variables (access-control-request-headers, access-control-request-method, etc.).  Any thoughts on what I might be missing?
Code:

[Assistant]
Text:Eric, can you paste the full request that you’re sending to Realm (including all the headers, body, etc)?We display an example of how to use Apollo Client with GraphQL here if you’re using GraphQL in the browser - realm-graphql-apollo-react/index.js at master · mongodb-university/realm-graphql-apollo-react · GitHub
Code:

[Assistant]
Text:
Unacceptable.
AWS, here we come
Code:

[Assistant]
Text:G’Day, @Jason_Steele,I acknowledge your frustration   Is there something we can help you with? Could you please share your use case and what you are looking for?Could we discuss a win-win solution for you and us? Cheers,
Code:

[Assistant]
Text:I’m having the same issue, but with Vue 3 not React and trying to use a apiKey. I’m trying to find code examples of how to get it to work but i’m not finding anything. Is there any examples for realm-graphql-apollo-vue3?
Code:

[Assistant]
Text:Hi guys! I am currently considering a Mongodb Atlas stack, but this issue looks like a deal breaker:This is a token example:This is the JWKs URI: https://beez-monitor.eu.auth0.com/.well-known/jwks.jsonMy app is https://eu-central-1.aws.data.mongodb-api.com/app/beez-app-loehxIs there a workaround, or am I doing something wrong?
Code:eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6IldvTmpxMlR5dzZJWWhlR0FhblFHTyJ9.eyJpc3MiOiJodHRwczovL2JlZXotbW9uaXRvci5ldS5hdXRoMC5jb20vIiwic3ViIjoiYXV0aDB8NjQ2Y2MyYWI2NjdkY2JjODk3ZjYyNzEyIiwiYXVkIjoiYmVlei1hcHAtbG9laHgiLCJpYXQiOjE2ODUxODY2OTUsImV4cCI6MTY4NTI3MzA5NSwiYXpwIjoiMXBuT1o3OFRybXlkSU1yMENMZGE2Q01GS1FQVkpqZ0QifQ

[Assistant]
Text:Never mind, figured it out: as @Pavel_Duchovny mentioned, I cannot use my custom JWT from Auth0, I need to exchange it first for a Mongodb Realm token, by making a request to https://eu-central-1.aws.realm.mongodb.com/api/client/v2.0/app/app-name-here/auth/providers/custom-token/login with the payload {“token”:} – this is sort of documented here: https://www.mongodb.com/docs/atlas/app-services/users/sessions/#authenticate-a-userYou guys could save the world a lot of time spent troubleshooting this if you would:(either of them would do  )
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Job for mongod.service failed because the control process exited with error code. See "systemctl status mongod.service" and "journalctl -xe" for details and tags: []

[User]
Text:i am on centos 7 when i trying to run sudo service mongod start i got an error like  this
Code:sudo service mongod startJob for mongod.service failed because the control process exited with error code. See "systemctl status mongod.service" and "journalctl -xe" for details.

[Assistant]
Text:The first thing to do when you get an error message that saysSee "systemctl status mongod.service" and "journalctl -xe" for details.is to see “systemctl status mongod.service” and “journalctl -xe” for details.Even us, we cannot help you without the details.
Code:See "systemctl status mongod.service" and "journalctl -xe" for details.

[Assistant]
Text:Any news on the details?  If you were able to fix your issue by following the advice, please mark my post as the solution so that others follow your steps.
Code:

[Assistant]
Text:@Brijesh_Kalkani, please follow up on your post.  Did you follow the tip I provided in my post?  Did it help you find a solution?  If so please mark it as the solution. This will help this forum to be efficient.
Code:

[Assistant]
Text:Hi,
you can see what are the errors, by typing in the shell:
systemctl status mongod.service
journactl -xe
tail -f /path/to/log_file/mongod.logRegards
Code:

[Assistant]
Text:I had the same error as OP and this is the output of journactl -xe:
Code:journactl -xeJan 18 15:31:33 rylan-ThinkPad-E590 kernel: CPU5: Core temperature/speed normal
Jan 18 15:31:33 rylan-ThinkPad-E590 kernel: CPU1: Core temperature/speed normal
Jan 18 15:31:33 rylan-ThinkPad-E590 kernel: CPU1: Package temperature/speed normal
Jan 18 15:31:33 rylan-ThinkPad-E590 kernel: CPU5: Package temperature/speed normal
Jan 18 15:31:33 rylan-ThinkPad-E590 kernel: CPU0: Package temperature/speed normal
Jan 18 15:31:33 rylan-ThinkPad-E590 kernel: CPU6: Package temperature/speed normal
Jan 18 15:31:33 rylan-ThinkPad-E590 kernel: CPU4: Package temperature/speed normal
Jan 18 15:31:33 rylan-ThinkPad-E590 kernel: CPU2: Package temperature/speed normal
Jan 18 15:31:33 rylan-ThinkPad-E590 kernel: CPU7: Package temperature/speed normal
Jan 18 15:31:33 rylan-ThinkPad-E590 kernel: CPU3: Package temperature/speed normal
Jan 18 15:31:38 rylan-ThinkPad-E590 slack.desktop[14549]: [01/18/22, 15:31:38:424] info: [RTM] (T2BRHD5EC) Processed 1 user_typing event(s) in channel(s) D02F5BU9B18 over 6.70ms
Jan 18 15:31:43 rylan-ThinkPad-E590 slack.desktop[14549]: [01/18/22, 15:31:43:710] info: [MSG-SHARED-RTM] (TC98Y5AQ3) dispatch to single store 1 msgs
Jan 18 15:31:43 rylan-ThinkPad-E590 slack.desktop[14549]: [01/18/22, 15:31:43:719] warn: (TC98Y5AQ3) Notification (message) suppressed because:
Jan 18 15:31:43 rylan-ThinkPad-E590 slack.desktop[14549]: Channel is muted
Jan 18 15:31:43 rylan-ThinkPad-E590 slack.desktop[14549]: [01/18/22, 15:31:43:720] info: [RTM] (TC98Y5AQ3) Processed 1 message:bot_message event(s) in channel(s) D02D5RQ95LL over 21.60ms
Jan 18 15:31:43 rylan-ThinkPad-E590 slack.desktop[14549]: [01/18/22, 15:31:43:780] info: [COUNTS] (TC98Y5AQ3) Updated unread_cnt for D02D5RQ95LL: 894
Jan 18 15:31:43 rylan-ThinkPad-E590 slack.desktop[14549]: [01/18/22, 15:31:43:943] info: [RTM] (T2BRHD5EC) Processed 1 user_typing event(s) in channel(s) D02F5BU9B18 over 7.30ms
Jan 18 15:31:46 rylan-ThinkPad-E590 sudo[13424]:    rylan : TTY=pts/1 ; PWD=/home/rylan/Documents/physics-benchmarking-neurips2021/experiments/dominoes_redyellow_pilot ; USER=root ; COMMAND=/bin/systemctl start
Jan 18 15:31:46 rylan-ThinkPad-E590 sudo[13424]: pam_unix(sudo:session): session opened for user root by (uid=0)
Jan 18 15:31:46 rylan-ThinkPad-E590 systemd[1]: Starting LSB: An object/document-oriented database...
-- Subject: Unit mongodb.service has begun start-up
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit mongodb.service has begun starting up.
Jan 18 15:31:46 rylan-ThinkPad-E590 mongodb[13427]:  * Starting database mongodb
Jan 18 15:31:47 rylan-ThinkPad-E590 mongodb[13427]:    ...fail!
Jan 18 15:31:47 rylan-ThinkPad-E590 systemd[1]: mongodb.service: Control process exited, code=exited status=1
Jan 18 15:31:47 rylan-ThinkPad-E590 systemd[1]: mongodb.service: Failed with result 'exit-code'.
Jan 18 15:31:47 rylan-ThinkPad-E590 systemd[1]: Failed to start LSB: An object/document-oriented database.
-- Subject: Unit mongodb.service has failed
-- Defined-By: systemd
-- Support: http://www.ubuntu.com/support
--
-- Unit mongodb.service has failed.

[Assistant]
Text:This is my output from looking at the first few lines of /var/log/mongodb/mongodb.log:
Code:/var/log/mongodb/mongodb.log2022-01-18T15:21:05.419-0800 I CONTROL  [initandlisten] MongoDB starting : pid=6332 port=27017 dbpath=/var/lib/mongodb 64-bit host=rylan-ThinkPad-E590
2022-01-18T15:21:05.419-0800 I CONTROL  [initandlisten] db version v3.6.3
2022-01-18T15:21:05.419-0800 I CONTROL  [initandlisten] git version: 9586e557d54ef70f9ca4b43c26892cd55257e1a5
2022-01-18T15:21:05.419-0800 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.1.1  11 Sep 2018
2022-01-18T15:21:05.419-0800 I CONTROL  [initandlisten] allocator: tcmalloc
2022-01-18T15:21:05.419-0800 I CONTROL  [initandlisten] modules: none
2022-01-18T15:21:05.419-0800 I CONTROL  [initandlisten] build environment:
2022-01-18T15:21:05.419-0800 I CONTROL  [initandlisten]     distarch: x86_64
2022-01-18T15:21:05.419-0800 I CONTROL  [initandlisten]     target_arch: x86_64
2022-01-18T15:21:05.419-0800 I CONTROL  [initandlisten] options: { config: "/etc/mongodb.conf", net: { bindIp: "127.0.0.1", unixDomainSocket: { pathPrefix: "/run/mongodb" } }, storage: { dbPath: "/var/lib/mongodb", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongodb.log" } }
2022-01-18T15:21:05.420-0800 I STORAGE  [initandlisten]
2022-01-18T15:21:05.420-0800 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2022-01-18T15:21:05.420-0800 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2022-01-18T15:21:05.420-0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=7429M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
2022-01-18T15:21:06.112-0800 I CONTROL  [initandlisten]
2022-01-18T15:21:06.112-0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2022-01-18T15:21:06.112-0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2022-01-18T15:21:06.112-0800 I CONTROL  [initandlisten]
2022-01-18T15:21:06.113-0800 I STORAGE  [initandlisten] createCollection: admin.system.version with provided UUID: 59618846-6a71-4606-b435-ec583269d327
2022-01-18T15:21:06.120-0800 I COMMAND  [initandlisten] setting featureCompatibilityVersion to 3.6
2022-01-18T15:21:06.124-0800 I STORAGE  [initandlisten] createCollection: local.startup_log with generated UUID: c5e16804-4ad9-4bf1-93e7-0282622efc94
2022-01-18T15:21:06.130-0800 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/var/lib/mongodb/diagnostic.data'
2022-01-18T15:21:06.130-0800 I NETWORK  [initandlisten] waiting for connections on port 27017
2022-01-18T15:22:27.230-0800 I CONTROL  [signalProcessingThread] got signal 15 (Terminated), will terminate after current cmd ends
2022-01-18T15:22:27.231-0800 I NETWORK  [signalProcessingThread] shutdown: going to close listening sockets...
2022-01-18T15:22:27.231-0800 I NETWORK  [signalProcessingThread] removing socket file: /run/mongodb/mongodb-27017.sock
2022-01-18T15:22:27.231-0800 I FTDC     [signalProcessingThread] Shutting down full-time diagnostic data capture
2022-01-18T15:22:27.232-0800 I STORAGE  [signalProcessingThread] WiredTigerKVEngine shutting down
2022-01-18T15:22:27.340-0800 I STORAGE  [signalProcessingThread] shutdown: removing fs lock...
2022-01-18T15:22:27.340-0800 I CONTROL  [signalProcessingThread] now exiting
2022-01-18T15:22:27.340-0800 I CONTROL  [signalProcessingThread] shutting down with code:0

[Assistant]
Text:Hey there!
This is what i got. Could you help me out here?
image919×153 19 KB

Code:

[Assistant]
Text:The first thing to do is to look at the log.
Code:

[Assistant]
Text:I could see that the logs folder from /var/log was deleted and this caused the db to shut down.
The below steps resolved the issue -
- switch to root user “sudo su”
- cd /var/log/
- mkdir mongodb
- chown -R mongod:mongod mongodb
- systemctl start mongod.service
Code:

[Assistant]
Text:Hi Steeve,I’m again facing issues with MongoDB. This time its a different error. I think its the space issue.
image1364×670 174 KB

Code:

[Assistant]
Text:Check free space on your /var/log mount point
df -k /var/log
df -i /var/log
Clean unnecessary files
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Find cause of write conflict and tags: ['java', 'transactions']

[User]
Text:This is using the reactive java driver: On my local dev machine I just saw the following error:`com.mongodb.MongoCommandException: Command failed with error 112 (WriteConflict): ‘WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction.’ on server 127.0.0.1:27017.The full response is:{“errorLabels”: [“TransientTransactionError”], “operationTime”: {"$timestamp": {“t”: 1602354840, “i”: 5}}, “ok”: 0.0, “errmsg”: “WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction.”, “code”: 112, “codeName”: “WriteConflict”, “$clusterTime”: {“clusterTime”: {"$timestamp": {“t”: 1602354840, “i”: 5}}, “signature”: {“hash”: {"$binary": “AAAAAAAAAAAAAAAAAAAAAAAAAAA=”, “$type”: “00”}, “keyId”: {"$numberLong": “0”}}}}`I also checked the MongoDB log, but there wasn’t a single entry at the timestamp when the transaction failed. Is there any way to find out which query has caused the transaction to fail?
Code:

[Assistant]
Text:What is the write concern for your write operation(s)?
Code:

[User]
Text:write concernWriteConcern is the default  - w: 1 afaik
Code:

[Assistant]
Text:Just because it’s easier than thinking, can you try “majority”?
Code:

[User]
Text:umm, just out of curiosity: why would that change anything if the driver only connects to a single node replica set?However, I can unfortunately not try that easily, because I haven’t seen that exception ever since. Tried to replicate, doing the same things, but it never showed up again so far.Our web clients (browser) connect to a web socket, which modifies the current user. There is a chance that this operation took place when a transaction (which also modifies the user document) was still running. Just a vague guess, but I haven’t found anything how I could check if this really is what has happened.
Code:

[Assistant]
Text:Hi @st-h …I was looking for easy first debugging steps, thinking along the lines of “If this happens a lot maybe it’s the volume of writes and there’s an inconsistency …”But what you say … that there exists the possibility of a transaction interfering with another write … well, if you know that’s a possibility in your application, it certainly sounds like the first thing to look at.Perhaps your code should handle exceptions and retry writes a few times in these instances.
Code:

[User]
Text:thanks for your reply. It’s not happening a lot. I observed it running the full stack on my local machine, with me being the only active user (that’s why I am worried about it). I am used to being able to lookup what the cause of an aborted transaction is due to using mysql before. It’s quite a surprise to find out this seems to not be possible at all with MongoDB.Do you by any chance know more details about how the java drivers handle transactions? I asked a related question quite a while ago (without any replies), as there is nothing mentioned in the docs: How to efficiently handle transactions with the reactive java driver / clarifications on apiHowever, the main issue here is that I would like to confirm if my suspicion is accurate - however, it seems that’s currently not possible at all - as there seems to be no way to find out why exactly a transaction failed?
Code:

[Assistant]
Text:@st-h I do no know more details about the Java drivers or how to find the cause of an aborted transaction.It’s quite a surprise to find out this seems to not be possible at all with MongoDB.Yes, I ran into a similar frustration. I had 3 tables to which I added validation. 2 worked; 1 did not. I searched and determined definitively there’s no way to cause MongoDB to log why a validation failed. There’s an open issue to add that feature.
Edit: I did figure out my problem after exhaustive trial-and-error … a misspelling in one place My takeaway is that, as good as MongoDB is, and I like it and use it daily, it’s not really “enterprise-ready”. It doesn’t have the completeness and stability of the mature relational products.
Code:

[Assistant]
Text:This might give a better insight:Transactions are new in MongoDB but have been existed in SQL databases for more than 30 years. Transactions are used to maintain…
Reading time: 5 min read

Code:

[Assistant]
Text:Hi, may I know if you found a solution to the WriteConflict error?
Code: