[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Mongod command is aborting after fassert() failure and tags: []

[User]
Text:Whenever I run the command mongod, I am getting this output.I tried deleting the file /tmp/mongodb-27017.sock and after restarting the mongod by sudo systemctl start mongod several times. However it is happening again and again. What should I do?
Code:mongod2020-06-22T10:18:36.490+0530 I  CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
2020-06-22T10:18:36.708+0530 W  ASIO     [main] No TransportLayer configured during NetworkInterface startup
2020-06-22T10:18:36.714+0530 I  CONTROL  [initandlisten] MongoDB starting : pid=12739 port=27017 dbpath=/data/db 64-bit host=mikhil-HP-Laptop-15-bs0xx
2020-06-22T10:18:36.714+0530 I  CONTROL  [initandlisten] db version v4.2.8
2020-06-22T10:18:36.714+0530 I  CONTROL  [initandlisten] git version: 43d25964249164d76d5e04dd6cf38f6111e21f5f
2020-06-22T10:18:36.714+0530 I  CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.1.1  11 Sep 2018
2020-06-22T10:18:36.714+0530 I  CONTROL  [initandlisten] allocator: tcmalloc
2020-06-22T10:18:36.714+0530 I  CONTROL  [initandlisten] modules: none
2020-06-22T10:18:36.714+0530 I  CONTROL  [initandlisten] build environment:
2020-06-22T10:18:36.714+0530 I  CONTROL  [initandlisten]     distmod: ubuntu1804
2020-06-22T10:18:36.714+0530 I  CONTROL  [initandlisten]     distarch: x86_64
2020-06-22T10:18:36.714+0530 I  CONTROL  [initandlisten]     target_arch: x86_64
2020-06-22T10:18:36.715+0530 I  CONTROL  [initandlisten] options: {}
2020-06-22T10:18:36.715+0530 E  NETWORK  [initandlisten] Failed to unlink socket file /tmp/mongodb-27017.sock Operation not permitted
2020-06-22T10:18:36.715+0530 F  -        [initandlisten] Fatal Assertion 40486 at src/mongo/transport/transport_layer_asio.cpp 684
2020-06-22T10:18:36.715+0530 F  -        [initandlisten]

***aborting after fassert() failure
/tmp/mongodb-27017.socksudo systemctl start mongod

[Assistant]
Text:I suspect that a previous instance of mongod was started with the root user.  As a result the file is now owned by root and cannot be deleted by your current user or the mongod user.
Code:

[Assistant]
Text:i am having the same problem, do you have any suggestion?
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Mongoimport command not found for MongoDB 4.4.0 server installation on macOS and tags: []

[User]
Text:Hi,
I tried importing a json file to my Mongodb atlas by running a mongoimport… but got a message that states: zsh: command not found: mongodb. My Mongodb version is 4.4.0, My OS is Catalina. When I ran ls “$(which mongo | sed ‘s/mongo//’)” | grep mongo, I got a list of mongo, mongod, mongos. I have also tried reinstalling and I keep getting the same responses. Please help.
Code:

[Assistant]
Text:Hello @Eze_Amadi,You can download the MongoDB tools separately at Download Database Tools. As of MongoDB 4.4, the server tools are also packaged separately and  have different versioning.Also, the documentation for mongoimport is found at a different location (as Database Tools), not as part from the server documentation: mongoimport.
Code:mongoimport

[Assistant]
Text:Hi @Eze_Amadi,As Prasad noted, the Database Tools are now separately versioned and installed as of the MongoDB 4.4 server release.The macOS installation guide and MongoDB Homebrew Tap should be updated to reflect this change, including information on how to install these tools if needed.I have raised a few related improvement suggestions you can watch and upvote:DOCS-13806: Add note on installing Database Tools to the 4.4 macOS installation guideSERVER-50089: MongoDB Homebrew Tap should include information on installing Database ToolsIn the interim, you can download the Database Tools via the MongoDB Download Centre as Prasad suggested.For more background on this change, please see Separating Database Tools from MongoDB Server (July, 2020).Regards,
Stennie
Code:

[Assistant]
Text:SERVER-50089: MongoDB Homebrew Tap should include information on installing Database ToolsI just noticed there is already a pull request in progress to update the Homebrew README.The following worked to install the latest version of the Database Tools:Regards,
Stennie
Code:brew tap mongodb/brew
brew install mongodb-database-tools

[User]
Text:It worked. Thanks to everyone that helped with this. I really appreciate!
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Index on Date column and tags: ['indexes']

[User]
Text:I have a Timestamp field in my documents for one of the collection. Is it a good idea to have a index on the Timestamp column ? I have a requirement in which I need to retrieve the documents between the certain dates. Trying to figure out the best way to retrieve the documents.Thank you,
Jason
Code:

[Assistant]
Text:Hello @Jason_Widener,Is it a good idea to have a index on the Timestamp column ?Yes, it is generally a good idea to have an index on a field used in a query criteria. This is really useful when there are a large number of documents (e.g., a million) in the collection. The index will be used to run the query fast.An index created on one field is called as a Single Field Index. This index is applied on a query with condition to retrieve documents for a range of values (e.g., between two Timestamp values) efficiently.Typically, you use a query like this to get the documents for a range of values:db.collection.find( { timestamp: { $gte: <start_value>, $lte: <end_value> } } )References:
Code:db.collection.find( { timestamp: { $gte: <start_value>, $lte: <end_value> } } )

[User]
Text:Thanks for your response @Prasad_Saya . Yes it is good idea to have index but is it good idea to have an index on Timestamp field/column ? Because anyways timestamp field is mostly going to be unique in all the documents. Unless two documents were inserted at exact same date and time.
Code:

[Assistant]
Text:is it good idea to have an index on Timestamp field/column ? Because anyways timestamp field is mostly going to be unique in all the documents. Unless two documents were inserted at exact same date and time.Timestamp is just a field in a document. You create an index on a field for the purpose of performance of queries and sort operations. Whether field is unique value or not, and the type of data (can be date, timestamp, string, number, etc) doesn’t matter.  Indexing and field properties are two different things.If your application requires that the field be unique, then an index of type unique can be created (see Unique Indexes).
Code:

[User]
Text:Awesome, thanks @Prasad_Saya
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Case insensitive query Regex (startsWith) and tags: []

[User]
Text:Hello there,using mongo since approx. 2 months and so far I’m quite happy, however we stumbled upon our first real problem.
One of our main use cases is a query e.g. by name. By default, the server should search case insensitively and with startsWith, so I’m doing sg. like this:While this delivers the correct results, it is very inefficient. This is stated in the documentation and I can validate this by looking at the explain-stats (it uses the index but scans ALL documents).An index with a collation strength of 1 or 2 does not help my case, because then I still loose the startsWith capability.Normalizing data on the way in is not really an option, unfortunately.I’m quite baffled this does not really work, I think i must be missing something. That is something that is very easy to do in most common sql environments by using a simple LIKE qiery along with a functional index (toLowerCase or sg. similar).Thank you!
Code:{"name.family" : /^christiansen/i}

[Assistant]
Text:Hi @Johannes_B,One of the possible solutions to your problem is a use of a text index in a combination of an additional $and expression.There are two possible ways to write this query once you indexed the field:This combination solve the insesetive search and the startWith in 2 effortsLet me know if that helps.Pavel
Code:{"name.family" : /^christiansen/}{"name.family" : /^christiansen/}

[User]
Text:Hello Pavel,thank you for your quick answer!
Not sure I understand it though  Maybe my example was a little misleading.In short, the term /^chri/ should deliver “Christiansen” in the resultset.For solution
1: As far as I understood the documentation,  regex with a text search does not work?
2: The collation indices don’t go well with regex searches, do they?It would be great if you could give me a short example.Thank you!
Code:

[Assistant]
Text:Hi @Johannes_B,Ok thank you for explaining in more detail. My solution was relevant for the first example when the full name is provided. This is since text searches are based on words or delimited words. Therefore with a text index you cant search a partial expression.I suggest exploring our Atlas search which comes to solve those problems on Atlas:Use MongoDB Atlas Search to customize and embed a full-text search engine in your app.Learn how to use a regular expression in your Atlas Search query.The original example was considering the following documents:Where the following aggregation will find the needed document:This will utilize the text index.Best regards
Pavel
Code:{ _id: ObjectID("5f2949ebbdee40880a3db0c0"),
  name: { first: 'David', family: 'Christiansen' } }
{ _id: ObjectID("5f294e2bbdee40880a3db0c1"),
  name: { first: 'David', family: 'Owen-Christiansen' } }
db.users.aggregate([{$match: {
  $text :{ "$search" : "christiansen" }
}}, {$match: {
  "name.family" : /^christiansen/i
}}])

[
    name: { first: 'David', family: 'Christiansen' } } ]


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Data directory C:\data\db\ not found and tags: ['installation']

[User]
Text:I tried to install MongoDB as a service and it runs through even with changing the directories for log and data to D:\develop\DB\MongoDb\Server\4.2…
When I try to run mongod from the command line I always get:
exception in initAndListen: NonExistentPath: Data directory C:\data\db\ not found. Create the missing directory or specify another path using (1) the --dbpath command line option, or (2) by adding the ‘storage.dbPath’ option in the configuration file., terminatingTrying to run tghe service as a domain user results in an error dialog.I wonder why MongoDb complains about C:}data\db.
Code:

[Assistant]
Text:How did you start mongod from command line?
If you just run mongod without dbpath it tries to start on default dir C:\data\db
So check if dir exists or not
Code:

[User]
Text:I just typed ‘mongod’ in the command line because in mongod.cfg I have:
Code:# mongod.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# Where and how to store data.
storage:
  dbPath: D:\develop\DB\MongoDB\Server\4.2\data
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path:  D:\develop\DB\MongoDB\Server\4.2\log\mongod.log

# network interfaces
net:
  port: 27017
  bindIp: 127.0.0.1

#processManagement:

#security:

#operationProfiling:

#replication:

#sharding:

## Enterprise-Only Options:

#auditLog:

#snmp:

[Assistant]
Text:When you are invoking mongod manually it is not using this default config file
It is looking for C:\ data\db.Since dir does not exist it is failingThe config file which you have pasted is used by default mongod which runs as service on WindowsTry to give a different port and a different path and see.It will work
Code:

[Assistant]
Text:I just typed ‘mongod’ in the command line@Dirk_Ulrich you need to point to the config file by using something like the following:orAs @Ramachandra_Tummala states, running just mongod, the process will use the default data values.
Code:mongod -f c:\path\to\mongod.conf
mongod --config c:\path\to\mongod.conf
mongod

[User]
Text:Both versions result in a non-responding prompt in the command line.
As you said, I installed Mongod already as a Windows service. I stopped it and tried to start mongod as you suggested but nothing happened.
Code:

[User]
Text:This happens also when I started CMD as admin.
Code:

[Assistant]
Text:It is not a non responsive prompt
Most likely your mongod is up and it is expected behaviour in WindowsYou have to open another cmd prompt and connect to mongodI am trying to understand your requirement
There was no need to stop the mongod which was running as service
This is default mongod which runs on prot 27017
All you have to do is connect by issuing mongoIf you want to start another mongod you have to use different port and path
That is what i meant in my reply
You should not change the default config fileCreate your own config file or you can start from command line alsomongod --port xyz --dbpath --logpathNote:You cannot have two mongods running on same portPlease go through mongo documentation for more details
Code:

[User]
Text:Well, I copied the original config file and renamed it from mongod.cfg to mongod.conf and changed port:Both config files are in the same directory…but that shouldn’t cause problems, should it?
Still I have a non responding prompt.
Did I understand you right, that MongoDb uses C:\data\db as default? …how does this work with the service running properly under windows althought this directory doesn’t exist?
Code:# mongod.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# Where and how to store data.
storage:
  dbPath: D:\develop\DB\MongoDB\Server\4.2\data
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path:  D:\develop\DB\MongoDB\Server\4.2\log\mongod.log

# network interfaces
net:
  port: 27027
  bindIp: 127.0.0.1

#processManagement:

#security:

#operationProfiling:

#replication:

#sharding:

## Enterprise-Only Options:

#auditLog:

#snmp:

[Assistant]
Text:D:\develop\DB\MongoDB\Server\4.2\dataWhen it is running as service it is using default config file and using this path-D:\develop\DB\MongoDB\Server\4.2\dataAlso it runs in backgroundBut when you run it manually it looks for C:\data\db
and it runs in foreground
That’s why your session appears to be hung
So you have to open another cmd prompt and connect using mongo --port your_portWhat exactly you mean by unresponsive prompt
It would have given some messageConfig file can be at same location or a different location but when you start mongod give the full path and change the dirpath to new path
You changed the port but left the dbpath same as the one being used when it runs as serviceDid you try a simple command line method?mongod --dbpath your_homedir --port 28001 --logpath your_homedir/mongod.logYou can try above by config file alsomongod -f path_to_your_homedir mongod.conf
Here i am assumng your config file resides in your home dir
Code:D:\develop\DB\MongoDB\Server\4.2\data


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Issues connecting to Atlas Database and tags: ['java']

[User]
Text:First of all - I am using MongoDB java driver 3.12.2I have created an project on atlas (using the free plan, so shared cluster) and am having trouble connecting to it using the java driver.Currently, I code I am using for connecting looks like this:And that works completely fine. However, when I try to search for a document in the database using code which looks like this:I get a very long exception:com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches com.mongodb.client.internal.MongoClientDelegate$1@7e32e7b1. Client view of cluster state is {type=REPLICA_SET, servers=[{address:27017=excelerate-shard-00-01.svfgy.mongodb.net, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names matching IP address 18.158.163.233 found}, caused by {java.security.cert.CertificateException: No subject alternative names matching IP address 18.158.163.233 found}}, {address:27017=excelerate-shard-00-00.svfgy.mongodb.net, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names matching IP address 18.185.151.83 found}, caused by {java.security.cert.CertificateException: No subject alternative names matching IP address 18.185.151.83 found}}, {address:27017=excelerate-shard-00-02.svfgy.mongodb.net, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: Exception sending message}, caused by {javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: No subject alternative names matching IP address 3.122.218.128 found}, caused by {java.security.cert.CertificateException: No subject alternative names matching IP address 3.122.218.128 found}}]My initial thoughts were that this was an authentication issue - so I triple checked that my network access was set to “access from anywhere”, username and password were correct.Another small thing I took notice of was that even if your password is incorrect in the connection URI, the MongoClients.create won’t throw an exception.
Also I tried connecting to the database in the mongo shell, and that seems to work just fine, which led me to think that this is an issue with the driver (which is why this is in the drivers section and not atlas section).I have done numerous google searches to figure this out, and I can’t find anyone with the exact exception I got.Any help is greatly appreciated, and if you need more information I will add that as soon as I get a chance.
Code:MongoClient mongoClient = MongoClients.create("mongodb+srv://excel:<password>@excelerate.svfgy.mongodb.net/database?retryWrites=true&w=majority");
database = mongoClient.getDatabase("database");
collection = database.getCollection("collection-1");
database.getCollection().find(Filters.eq("user-id", this.userId)).first();

[Assistant]
Text:Hi @Excel8392,Considering the following cause:I believe your issue is that the java key store cannot locate the atlas public CA. This is required as Atlas traffic requires SSL.Please verify that the latest certificate is pushed in your java store:Also look on the java consideration on that page.Let me know if that helps.Pavel
Code:caused by {java.security.cert.CertificateException: No subject alternative names matching IP address 18.158.163.233 found}},

[User]
Text:Thanks for the quick response!I do not have a Let’s Encrypt certificate set up, and the process of setting one up seems to be a little tedious. Is it possible to either use a different type of certificate, or no certificate at all?
Sorry if the answer to my question is obvious, I am new to using MongoDB and am still trying to figure everything out.
Code:

[Assistant]
Text:Hi @Excel8392,Atlas requires this certificate for the SSL encryption.For latest Java runtime it should be present in the key store, but I suspect you are using an older version:Let’s Encrypt isn’t present in the default trust store for Java version 7 prior to the 7u111 update, or Java version 8 prior to the 8u101 update. Use a Java release after 19 July 2016.Please ensure your Java client software is up-to-date. The latest Java versions are strongly recommended for many improvements beyond these new Certificate Authority requirements for our TLS certificates.Anyway it should not take so much time to configure it on your machine.Please let me know if you have any additional questions.Best regards,
Pavel
Code:keytool -importcert -file <root-certificate-filename> -keystore </path/to/keystore/keystore.jks> -alias "Alias"
keytool -list -cacerts >certs.txt
 grep -i 'dst_root' certs.txt

[User]
Text:Thank you so much for the quick response, that fixed it.
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Mongodb compass- freeze on activating plugins and tags: ['compass']

[User]
Text:after installing compass, i ran the program but it only get to the point where it is “activating plugins” but never goes beyond this and the circle "loader keeps on turning but nothing happens … hours and hours can go by, i can restart my machine and still everytime i run this program it gets to the same point the does not go any further than “activating plugins” …someone please help!!!

Code:

[Assistant]
Text:Hi @Angelo_Hedley can you please provide a little more information:
Code:

[Assistant]
Text:Hello @Angelo_HedleyDo you see any errors when you click ALT+CTRL+I and check out the console tab on the right side?
grafik1280×210 23 KB
Michael
Code:

[User]
Text:gram but it only get to the point where it is “activating plugins” but never goes beyond this and the circle "loader keeps on turning but nothing happens … hours and hours can go by, i can restart my machine and still everytime i run this program it gets to the same point the does not go any further than “activating plugins” …HI DougIt only gets to the activating plugins then it does not go any further
Windows 7 i7 HP ELITEBOOK 8560p
version
1.21.0  (Stable )
Code:

[Assistant]
Text:HI there, I’m experiencing the same issue and look forward to getting a resolve. I’ve uninstalled and reinstalled Compass and still get the same result.
Code:

[Assistant]
Text:Hi @Charlie_Maru, we are looking into the problem but we are having some trouble reproducing it.Can I ask you to do a quick test? You should have all the Compass user preferences in %APPDATA%/MongoDB Compass. Can you delete that folder (make a copy of it in case you have something that you want to keep, like favorite connections or query history) and start Compass again?
Code:%APPDATA%/MongoDB Compass

[Assistant]
Text:Hi @Massimiliano_Marcon
I’ve done as you’ve suggested but it still stalls at “Activating Plugins”. It goes no further.

image1600×900 58.8 KB

Code:

[Assistant]
Text:Thank you for trying. That’s helpful to at least exclude one cause.
I will keep you posted as we make progress with this issue.
Code:

[Assistant]
Text:Hi, I am also having this problem!
Code:

[Assistant]
Text:I also have this issue…
Code:

[Assistant]
Text:Hi @Alfredo_N_A, @Gabriel_Timofti,Welcome to the community! If you are experiencing the same issue please provide some additional information to help us track down the problem:Thanks,
Stennie
Code:

[Assistant]
Text:Hi. Thx for Response.
version: 1.21.0 (Stable Community)
Os: Win 10 x 64
It’s a fresh install.I’ll keep updating the status
Code:

[Assistant]
Text:I just borrowed my son’s Windows 10 machine to test and Compass opens up just fine for me. I did notice that there are three packages that can be downloaded for Windows.I downloaded the EXE package. @Gabriel_Timofti, @Alfredo_N_A and @Charlie_Maru can you verify which package type you’re using? I wonder if it might be a problem with the MSI by chance.
Code:

[Assistant]
Text:Hi Doug,Thank you for your email.  So “what is a bear’s favorite drink? = koka-koala!”, I downloaded the MSI package.  Keep in mind I am following the MongoDB University slides at the moment, maybe that’s a great place to start.Semper fi,
Code:

[Assistant]
Text:Hi Stennie,
Mongodb 1.21.0
Win 10 x64
I’ve had mongo for some time, and only recently (3 weeks ago) it started to lock up when activating plugins
Code:

[Assistant]
Text:Hi @Charlie_Maru, Compass 1.21.0 was released on April 28th (about two weeks ago). Is it possible that you updated around that time? If so it’s possible that the update did something. If you didn’t update then hopefully the MongoDB engineers can help you track down what’s going on.
Code:

[Assistant]
Text:Last week, we released Compass 1.21.1. It should fix some error conditions as well as print some useful error messages in the console of the loading screen if something goes wrong. Could you please give that a try?Thank you!
Code:

[User]
Text:i was told to use the isolated edition instead until the problem is fixed. it works out ok but how will i know when the problem is fixed? who will let me know ?
Code:

[Assistant]
Text:@Angelo_Hedley would you be able to try 1.21.1 (non isolated edition)? We fixed some of the potential issues in there but we’ve had a hard time reproducing the issue in our windows boxes. It’d be great to get some feedback from the community on whether the problem is really solved.
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Using $group to group data by a field, and then $sort by field within nested array and tags: ['aggregation']

[User]
Text:I’m attempting to group sessions by location, and then sort the session objects within the nested Sessions array by StartTime. I’ve found a way to do this using an index on StartTime, which is used in the initial $match, but I’d like a way to explicitly specify my sorting intentions. Does anyone have a way I can do this?This how I would like the aggregation to work. Essentially the $sort stage here does nothing. Is there any way to $sort the objects pushed into Sessions in the $group stage by StartTime?
Code:db.getCollection('sessions').aggregate([
    {
      $match: {
          $and: [
            { "StartTime": { $gte: ISODate("2020-02-01T18:52:34.000Z") } },
            { "StartTime": { $lte: ISODate("2020-02-02T18:52:34.000Z") } }
          ]
      }
    },
    {
      $group: {
          _id: { id: "$locationId" },
          numberOfSessions: { $sum: 1 },
          Sessions: {
              $push: {
                  _id: "$_id",
                  SessionID: "$SessionID",
                  StartTime: "$StartTime",
                  EndTime: "$EndTime"
              }
          }
      }
    },
    {
      $match: {
          numberOfSessions: { $gte: 2 }
      }
    },
    { $sort: { 'Sessions.StartTime': 1 } },
    { $project: { StartTime: -1, Sessions: -1, numberOfSessions: -1 } }
  ],
  {
     allowDiskUse: true
  }
)

[Assistant]
Text:Hello, @Greg_Fitzpatrick-Bel!Let me show how to solve this by example.
Assume, we have this dataset:And this aggregation:Will provide use with this result:Notice, that session objects are inserted in arbitrary order, but aggregation returned sessions, ordered by startTime.
Code:db.sessions.insertMany([
  {
    _id: 'sesh-1',
    startTime: ISODate('2020-07-01T18:52:34.000Z'),
    endTime: ISODate('2020-08-01T18:52:34.000Z'),
    locationId: 'L2',
  },
  {
    _id: 'sesh-2',
    startTime: ISODate('2020-05-01T18:52:34.000Z'),
    endTime: ISODate('2020-06-01T18:52:34.000Z'),
    locationId: 'L1',
  },
  {
    _id: 'sesh-3',
    startTime: ISODate('2020-03-01T18:52:34.000Z'),
    endTime: ISODate('2020-04-01T18:52:34.000Z'),
    locationId: 'L1',
  },
  {
    _id: 'sesh-4',
    startTime: ISODate('2020-01-01T18:52:34.000Z'),
    endTime: ISODate('2020-02-01T18:52:34.000Z'),
    locationId: 'L2',
  },
]);
db.sessions.aggregate([
  {
    $match: {
      // match the docs anyhow you want
      // this $match stage will match all docs
      // from sessions collection
    }
  },
  {
    $group: {
      // group by $locationId
      _id: '$locationId',
      numberOfSessions: {
        $sum: 1,
      },
      sessions: {
        $push: {
          _id: '$_id',
          startTime: '$startTime',
          andTime: '$endTime',
        }
      }
    }
  },
  {
    // this is needed to sort items in $sessions array
    $unwind: '$sessions',
  },
  {
    $sort: {
      // specify $sessions sort params here
      'sessions.startTime': 1,
    }
  },
  {
    // this $group stage is needed, because we did
    // $unwind before
    $group: {
      _id: '$_id',
      numberOfSessions: {
        $first: '$numberOfSessions',
      },
      sessions: {
        $push: '$sessions',
      }
    }
  }
]).pretty();
[
  {
    "_id" : "L1",
    "numberOfSessions" : 2,
    "sessions" : [
      {
        "_id" : "sesh-3",
        "startTime" : ISODate("2020-03-01T18:52:34Z"),
        "andTime" : ISODate("2020-04-01T18:52:34Z")
      },
      {
        "_id" : "sesh-2",
        "startTime" : ISODate("2020-05-01T18:52:34Z"),
        "andTime" : ISODate("2020-06-01T18:52:34Z")
      }
    ]
  },
  {
    "_id" : "L2",
    "numberOfSessions" : 2,
    "sessions" : [
      {
        "_id" : "sesh-4",
        "startTime" : ISODate("2020-01-01T18:52:34Z"),
        "andTime" : ISODate("2020-02-01T18:52:34Z")
      },
      {
        "_id" : "sesh-1",
        "startTime" : ISODate("2020-07-01T18:52:34Z"),
        "andTime" : ISODate("2020-08-01T18:52:34Z")
      }
    ]
  }
]


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Why do indexes use B-Tree and not Binary Search Tree? and tags: ['indexes']

[User]
Text:https://university.mongodb.com/mercury/M201/2020_July_7/chapter/Chapter_2_MongoDB_Indexes/lesson/58ab5626d280e426708ee3da/lectureIn this Chapter it is mentioned Mongodb Uses B-Tree for Indexes. So i have a question Why it does not use Binary Search Tree because Insertion, deletion, searching of an element is faster in BINARY SEARCH TREE than BINARY TREE due to the ordered characteristics. So Does it Uses BST if no than Why?
Code:

[Assistant]
Text:Welcome to the community @Vaibhav_Parashar!It looks you are conflating a B-Tree with a simple Binary Tree: these are not the same data structures.A B-tree is a self-balancing generalisation of the Binary Search Tree (BST) data structure with better performance characteristics for use cases that work with large amounts of data. B-trees and BSTs have the same average Big-O complexity, but a BST’s worst case complexity for search, insert, and update is O(n)) versus B-tree’s O(log(n)).The above links include some helpful information on the differences and benefits of these data structures. B-trees are commonly used (and well-suited) for general database and filesystem indexing. There are some variants of B-trees (for example: B-tree category on Wikipedia) and differences in implementation specific to the architecture of different applications, but the high level concepts are similar.There are also some variations in how B-tree indexes may be used in different products. For example, MongoDB’s WiredTiger storage engine uses prefix compression for index values by default. Prefix compression still stores values in a general B-Tree structure, but reduces some of the repetitive details for more efficient index size. For an older (but still applicable) overview, see WiredTiger - how to reduce your MongoDB hosting costs.due to the ordered characteristicsB-trees are ordered, and the order of keys is important if you want to efficiently use an index to sort query results.Regards,
Stennie
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Problem with deserialization and tags: ['dot-net']

[User]
Text:Hi,I am quite new in MongoDB and having some problems with reading my data using CSharp driver. I have a class with a Point class field called “APoint”. The document is created and APoint is an object with X and Y values as integers. So far so good. However when I list the documents I get this error. I am sure I need to write custom serialization and deserialization code, put I couldn’t find a decent example.Any help is much appreciated.
Thanks
MGSystem.FormatException: ‘An error occurred while deserializing the APoint field of class WindowsFormsApp1.ProgramSettings: Value class System.Drawing.Point cannot be deserialized.’
Code:

[Assistant]
Text:Hi @mgtheone, and welcome to the forumI have a class with a Point class field called “APoint”. The document is created and APoint is an object with X and Y values as integers.This is likely because the registry doesn’t know how to deserialise System.Drawing.Point. You mentioned that the value of X and Y are integers, depending on your use case try to define the type as int. i.e.If you still encountering this issue, could you provide:Regards,
Wan.
Code:intpublic class MyPoint
{
    public int X {get; set;}
    public int Y {get; set;}
}

[User]
Text:Thanks Wan, I  actually found a solution. I wrote a custom serializer called “MyCustomPointSerializer”. I will submit my code now.
Thanks a lot.
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: "shard version not ok: version mismatch detected for" and tags: ['sharding']

[User]
Text:Good afternoon,MongoDB server version: 3.6.16 on RHEL/CentOS.I am trying to insert a record into a sharded collection, and I am getting a strange error I have Googled the heck out of.  To summarize, the insert says: “shard version not ok: version mismatch detected for MYDATABASE.MYCOLLECTION”.  What causes this?  I have tried to do a flushRouterConfig with no avail.The following is the log line I that I have changed to protect the innocent:2020-07-22T16:23:45.805+0000 I COMMAND  [conn1189005] command MYDATABASE.MYCOLLECTION command: insert { insert: “MYCOLLECTION”, bypassDocumentValidation: false, ordered: false, documents: 50, shardVersion: [ Timestamp(57023, 3), ObjectId(‘57c5ff81724c2e70c623e733’) ], lsid: { id: UUID(“fade92f1-0993-4db7-af03-1b6066628f8c”), uid: BinData(0, 30D9CB0F31D33F7912528ADD7F28D77AA3ADBBDF1B6E9C50BFB8163217CE97C8) }, $clusterTime: { clusterTime: Timestamp(1595435024, 596), signature: { hash: BinData(0, E82B2EB1A3E65A615144ABFD585CC0CB8DB8E2D0), keyId: 6817327097028018329 } }, $client: { driver: { name: “mongo-java-driver”, version: “3.9.1” }, os: { type: “Linux”, name: “Linux”, architecture: “amd64", version: “3.10.0-1127.el7.x86_64” }, platform: “Java/Oracle Corporation/1.8.0_181-b13", mongos: { host: “queryrouter:27018”, client: “xxx.xxx.xxx.xxx:49626", version: “3.6.16” } }, $configServerState: { opTime: { ts: Timestamp(1595435023, 177), t: 4071 } }, $db: “MYDATABASE” } ninserted:0 exception: shard version not ok: version mismatch detected for MYDATABASE.MYCOLLECTION ( ns : MYDATABASE.MYCOLLECTION, received : 57023|3||57c5ff81724c2e70c623e733, wanted : 57024|3||57c5ff81724c2e70c623e733 ) code:StaleConfig numYields:0 reslen:14363 locks:{ Global: { acquireCount: { r: 4, w: 2 } }, Database: { acquireCount: { r: 1, w: 2 } }, Collection: { acquireCount: { r: 1, w: 2 }, acquireWaitCount: { w: 1 }, timeAcquiringMicros: { w: 1432380 } } } protocol:op_msg 1432msit’s always on the same object ID, and we have over 2000-3000 errors daily.  Mongo cannot find the objectID:Also running below and restarting all the query routers did not fix it:db.adminCommand({ flushRouterConfig: “MYDATABASE.MYCOLLECTION”});Please assist.
Code:mongos> db.MYCOLLECTION.find({_id :
ObjectId("57c5ff81724c2e70c623e733" });

[Assistant]
Text:Hello @William_Crowell, welcome to the community!It would be helpful for others to read and respond to your question if you could apply the proper log  formatting to your post. Please add spacing to break up long blocks of text/logs and improve readability.
You may want to review the Getting Started guide form @Jamie which has some great additional tips and information.Concerning your actual problem: it seems that your run different versions.version mismatch detected for MYDATABASE.MYCOLLECTION ( ns : MYDATABASE.MYCOLLECTION, received : 57023|3||57c5ff81724c2e70c623e733, wanted : 57024|3||57c5ff81724c2e70c623e733 )You can check the shards mongodb.log and look for something like:requested shard version differs from config shard version for my_db.my_collection, requested version isCheers,
Michael
Code:

[User]
Text:Hey Michael,Thanks for your reply.  I just added a new line after each line to improve the readability, and I am providing a public gist link to it:https://gist.githubusercontent.com/wcrowell/e29eb4b98d1a1fd5e78ca713876813b6/raw/649f01af9d16d523271147de6b43e219ee5270ec/formatted.txtLet me know if I can do anything else to improve readability.I did not see the message: “requested shard version differs from config shard version for my_db.my_collection, requested version is”.Regards,Bill Crowell
Code:

[Assistant]
Text:I just checked last one week of the Mongo log. We do not have “requested shard version differs”
Code:

[Assistant]
Text:Hi @William_Crowell and @Fory_Horio,The errors indicate that at some point the sharding metadata was stale/out of sync.Usually invalidation the cache on the mongos/shards should solve this:Can you confirm the issue is gone now?Best regards,
Pavel
Code:

[User]
Text:Pavel,
Good morning.  Do you mean running: db.adminCommand({ flushRouterConfig: " PM_AUDIT.AUDIT " } )Or another method to invalidate the cache like this: https://docs.mongodb.com/manual/reference/command/invalidateUserCache/Thanks for your reply.Regards,Bill Crowell
Code:

[Assistant]
Text:Hi @William_Crowell,Yes I meant flushRouterConfig.Yes you can start with the collection level and escalateBest regards
Pavel
Code:

[User]
Text:Pavel,Thanks for your reply again.  This would need to be run on both the query routers, mongod database instances, and mongod configuration database instances?Regards,William Crowell
Code:

[Assistant]
Text:We have ran the db.adminCommand({ flushRouterConfig: "  PM_AUDIT.AUDIT  " } ) on all the QRTs and restarted mongos several times, but the error never disappear. A few months ago, we had a similar problem, at that time, flushing and restarting mongos cured the problem. But not this time. We tried several times over few days.
Code:

[Assistant]
Text:Hi @Fory_HorioHave you run it on all mongos instances and shards?What is the version of the cluster and the sharding distribution of this collection?Can you consider failover the shards?Best regards
Pavel
Code:

[Assistant]
Text:Yes, I did, several times on all the QRTs on all shards. Didn’t get solved.
Code:

[Assistant]
Text:We have two shards with two QRTs for each shard, total four QRTs. I tried one more round of flushing and restarting mongos against all four QRTs. The last error BEFORE the restarts is blow. We will see.2020-07-27T21:24:19.964+0000 I COMMAND  [conn1221010] command PM_AUDIT.AUDIT command: insert { insert: “AUDIT”, bypassDocumentValidation: false, ordered: false, documents: 50, shardVersion: [ Timestamp(60823, 3), ObjectId(‘57c5ff81724c2e70c623e733’) ], lsid: { id: UUID(“0cc98ccd-4699-48fd-b418-dac54bf66319”), uid: BinData(0, 30D9CB0F31D33F7912528ADD7F28D77AA3ADBBDF1B6E9C50BFB8163217CE97C8) }, $clusterTime: { clusterTime: Timestamp(1595885058, 260), signature: { hash: BinData(0, 7469BFFD3F6BAB1F6127C9839A57F6ADFC72AB01), keyId: 6817327097028018329 } }, $client: { driver: { name: “mongo-java-driver”, version: “3.9.1” }, os: { type: “Linux”, name: “Linux”, architecture: “amd64”, version: “3.10.0-1127.el7.x86_64” }, platform: “Java/Oracle Corporation/1.8.0_181-b13”, mongos: { host: “monqrt-east-1b:27018”, client: “10.1.2.121:48620”, version: “3.6.16” } }, $configServerState: { opTime: { ts: Timestamp(1595885057, 177), t: 4071 } }, $db: “PM_AUDIT” } ninserted:0 exception: shard version not ok: version mismatch detected for PM_AUDIT.AUDIT ( ns : PM_AUDIT.AUDIT, received : 60823|3||57c5ff81724c2e70c623e733, wanted : 60824|3||57c5ff81724c2e70c623e733 ) code:StaleConfig numYields:0 reslen:14363 locks:{ Global: { acquireCount: { r: 4, w: 2 } }, Database: { acquireCount: { r: 1, w: 2 } }, Collection: { acquireCount: { r: 1, w: 2 }, acquireWaitCount: { w: 1 }, timeAcquiringMicros: { w: 1214048 } } } protocol:op_msg 1214ms
Code:

[Assistant]
Text:After flushing and restarting ALL QRTs, the error came back again.2020-07-27T23:16:53.597+0000 I COMMAND  [conn1221486] command PM_AUDIT.AUDIT command: insert { insert: “AUDIT”, bypassDocumentValidation: false, ordered: false, documents: 50, shardVersion: [ Timestamp(60855, 3), ObjectId(‘57c5ff81724c2e70c623e733’) ], lsid: { id: UUID(“694c2e56-c025-48e9-9e60-e71248a44bd3”), uid: BinData(0, 30D9CB0F31D33F7912528ADD7F28D77AA3ADBBDF1B6E9C50BFB8163217CE97C8) }, $clusterTime: { clusterTime: Timestamp(1595891811, 296), signature: { hash: BinData(0, 786285F2A69138520A346EBF33FA7B3CFEA96DD2), keyId: 6817327097028018329 } }, $client: { driver: { name: “mongo-java-driver”, version: “3.9.1” }, os: { type: “Linux”, name: “Linux”, architecture: “amd64”, version: “3.10.0-1127.el7.x86_64” }, platform: “Java/Oracle Corporation/1.8.0_181-b13”, mongos: { host: “monqrt-east-1c:27018”, client: “10.1.3.191:41186”, version: “3.6.16” } }, $configServerState: { opTime: { ts: Timestamp(1595891811, 3), t: 4071 } }, $db: “PM_AUDIT” } ninserted:0 exception: shard version not ok: version mismatch detected for PM_AUDIT.AUDIT ( ns : PM_AUDIT.AUDIT, received : 60855|3||57c5ff81724c2e70c623e733, wanted : 60854|3||57c5ff81724c2e70c623e733 ) code:StaleConfig numYields:0 reslen:14363 locks:{ Global: { acquireCount: { r: 7, w: 3 } }, Database: { acquireCount: { r: 2, w: 3 } }, Collection: { acquireCount: { r: 2, w: 2, W: 1 }, acquireWaitCount: { W: 1 }, timeAcquiringMicros: { W: 1549896 } } } protocol:op_msg 1994ms
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Alias field in Realm function and tags: ['atlas-functions']

[User]
Text:Hello,I am trying to alias the _id field in a Realm function using the find() method.
According to the documentation, the second parameter in the find() method is a projection.I have a collection called students with the following fields
_id, firstName, lastName, accountIdI am attempting to alias the return using a projection
Here is a code snippet:Here are my results[
{"_id":“5e8fc09770f5e3f18c730a21”},{"_id":“5e8fc40470f5e3f18c730a22”},{"_id":“5e90838738eff556f0fa48d0”},{"_id":“5e9086a138eff556f0fa48d1”},{"_id":“5e908dba38eff556f0fa48d2”},{"_id":“5eea898c1a3f23cd2980e699”}]The alias did not work and I am also missing the other fields.Anyone have an idea?  I went through the Realm docs and the samples are very scarce
Code:const query = { };
const projection = { _id_:1, studentId: "$_id", };
await students.find(query, projection).toArray()

[Assistant]
Text:Hi Herb -Can you try putting quotes around “_id” and “studentId”? Also, to return any other field you will have to add it to your projection.For example, I tried something similar and got this snippet to work:
Code:exports = function(arg){

const mongodb = context.services.get("mongodb-atlas");
const itemsCollection = mongodb.db("db").collection("Cat");

  const query = { "name": "joey"};
  const projection = { "_id": 1, "StudentId": "$_id"};

return itemsCollection.find(query, projection)
  .toArray()
  .then(items => {
    console.log(`Successfully found ${items.length} documents.`)
    items.forEach(console.log)
    return items
  })
  .catch(err => console.error(`Failed to find documents: ${err}`))

};

[User]
Text:Hello Sumedha,I tried putting the quotes around the fields and still can’t get the alias to work
const projection = { “_id”: 1, “studentId”:"$_id", “firstName”:"$firstName",“lastName”:"$lastName",     “accountId”:"$accountId"};Unfortunately, the alias is not working.  I also tried aliasing other fields (ex: “first”:"$firstName")
Other alias fields don’t appear at all in the outputTo be clear, I am using a Realm ‘3rdPartly’ HTTP Get function, not Compass or the CLI{
[
{
“_id”: “5e8fc09770f5e3f18c730a21”,
“accountId”: “5e89f69d1c9d440000929b9a”,
“firstName”: “Nancy”,
“lastName”: “Doe”
},
{
Code:        "_id": "5e8fc40470f5e3f18c730a22",
        "accountId": "5e89f69d1c9d440000929b9a",
        "firstName": "Pat",
        "lastName": "Doe"
    },
    {
        "_id": "5e90838738eff556f0fa48d0",
        "accountId": "5e89f69d1c9d440000929b9a",
        "firstName": "Michael",
        "lastName": "Doe"
    },
    {
        "_id": "5e9086a138eff556f0fa48d1",
        "accountId": "5e89f69d1c9d440000929b9a",
        "firstName": "Sam",
        "lastName": "Doe"
    },
    {

        "_id": "5e908dba38eff556f0fa48d2",
        "accountId": "5e89f69d1c9d440000929b9a",
        "firstName": "Maria",
        "lastName": "Doe"
    },
    {

        "_id": "5f18c5e9c0764f3b76890023",
        "accountId": "5e89f69d1c9d440000929b9a",
        "firstName": "John",
        "lastName": "Doe"
    }
]

[Assistant]
Text:How are you testing your function? Is it with the Realm UI Console?Do you mind pasting your entire function snippet here as well.
Code:

[User]
Text:Sure.  Its a webhook in Realm.
I’m not actually using a Realm “Function” but a Realm 3rdParty HTTP webhook.Here is the code:/*exports = async function getStudents(payload, response) {
const {accountId  } = payload.query;
const db = context.services.get(“mongodb-atlas”).db(“usersDB”);
const students = db.collection(“students”);
console.log(getStudents accountId:${accountId});}
Code:getStudents accountId:${accountId}const query = { accountId:accountId };
//const projection = { "studentId": "$_id", };
const projection = { "_id": 1, "studentId":"$_id", "firstName":"$firstName","lastName":"$lastName",  "accountId":"$accountId"};

console.log ('getting students query:', JSON.stringify(query));

response.setHeader("Content-Type","application/json");

//await students.find(query).toArray()
await students.find(query, projection).toArray()
.then(result => {
    if(result) {
      response.setStatusCode(200);
      //response.setBody(`{"students":${result}`);
      response.setBody(`{"students":${JSON.stringify(result)}}`);
    }
    else {
      console.log("students not found:",JSON.stringify(result));
      response.setStatusCode(404);
      response.setBody(`{message:"No students not found for given criteria"}`);
    }
}).catch(err => {
  console.log("error getting students:",err);
  response.setStatusCode(500);
  response.setBody(`{error:${err}}`);
})

[Assistant]
Text:Hey Herb,I’m looking into why renaming aliases seems to work sometimes on Realm (e.g. my function), but not in your caseIn the meantime @wan has an alternative solution using a pipeline that should give you the same result.
Code:

[Assistant]
Text:Hi @Herb_Ramos,The alias did not work and I am also missing the other fields.If you’re looking to project a field under a different name, and also include other fields as well you should try Realm aggregation pipeline: project document fields. An example for your use case would be:Regards,
Wan.
Code:const pipeline = [
  { "$addFields": {"StudentId": "$_id" } }
  ]

  return students.aggregate(pipeline).toArray()
  .then(students => {
    console.log(`Successfully grouped purchases for ${students.length} customers.`)
    students.forEach(console.log);
    return students
  })
  .catch(err => console.error(`Failed to find documents: ${err}`));*/

[User]
Text:Thanks Wan. I will give this a try.   I am however curious if there is a bug based on my original request.
Code:

[Assistant]
Text:Hi @Herb_Ramos,I am however curious if there is a bug based on my original request.It is not a bug, and apologies for the confusion.If your Realm application is linked to MongoDB Atlas version 4.4 (currently available in beta in select regions only), you should be able to project field name alias using find() projection syntax. If you are using the current stable MongoDB Atlas version 4.2, you should use the aggregation pipeline to project field name aliases.Since your valid code function did not return the projection output aliases, I’m assuming that your MongoDB Atlas cluster is on version 4.2Regards,
Wan.
Code:find()

[User]
Text:Thanks Wan. I will mark this as a solution.  I am on 4.2.  I have not tried the projection on 4.4 and I am assuming it will work.  For now I will used the pipeline aggregate you suggested on 4.2.  Thanks again.
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Errors when installing MongoDB on Ubuntu 20.04 and tags: ['installation']

[User]
Text:i follow the official install instructionhttps://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/
Code:

[Assistant]
Text:Hi @Danh_LeYou may have skipped over this section:MongoDB 4.2 Community Edition supports the following 64-bit Ubuntu LTS (long-term support) releases on x86_64 architecture:See the commenst from @Stennie_X Ubuntu 20.04 support - #2 by Stennie_X
Mongodb error in Ubuntu 20.04 - #3 by Stennie_X
Code:

[User]
Text:Thanks for reply @chris i installed version 4.4 and follows some instructions on Stackoverflow which i dont even rememer what i did lol. but i made progress, i could active the mongo sever but could active the mongod command, every time i type mongod i got  Failed to unlink socket file /tmp/mongodb-27017.sock Operation not permitted  i trying to change the port… any advice i should know?
Code:mongodFailed to unlink socket file /tmp/mongodb-27017.sock Operation not permitted

[User]
Text:hi @chris just let you know. i got my error fixed. by sudo mkdir -p /data/db  and run mongodb with sudo mongod i works greet now, i am so happy its been four days thanks god.
Code:sudo mkdir -p /data/dbsudo mongod


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Storing strings in MongoDB larger than 16 MB and tags: []

[User]
Text:Hello,I would like to store strings larger than 16 MB into MongoDB. From the documentation I learned that there is GridFS that I can use to store files into the database.Here’s how my software works:So the file (Point 3) that the user can select will be converted to a string and then transfered to the server.In the documentation I couldn’t find a class method to store strings to GridFS. Is there a way I can store them into GridFS? Or is there a way I can store it normally in a collection? For example, can I set the size limit for data in MongoDB? Is that possible?
Code:

[Assistant]
Text:Hello, @Cryptnote!In the documentation I couldn’t find a class method to store strings to GridFS.You can check MongoDB driver’s tutorials for GridFS usage examples
Code:

[User]
Text:Well yes, I know the documentation. There’s no method for GridFS which you can use to store a large string.
Code:

[Assistant]
Text:I am a Java developer (am not familiar with PHP; I think you are using MongoDB PHP driver). I tried your issue of storing a string into GridFS collections. It works both ways, writing a string and retrieving it. Ofcourse, I used Java APIs. I can relate to what I did (code) in PHP, and I think a PHP developer can figure the rest.Writing to GridFS:Start from this topic: Uploading Files with Writable StreamsSee the bucket class function: MongoDB\GridFS\Bucket::uploadFromStreamThe function definition:uploadFromStream($filename, $source, array $options = [])The Readable stream is an input stream which is created from the input string; and this needs to be created in PHP code ($streamToUploadFrom). With Java, the stream is created as:
InputStream streamToUploadFrom = new ByteArrayInputStream(inputString.getBytes("UTF-8"));Write the string to the MongoDB GridFS collections:The function returns: The _id field of the metadata document associated with the newly created GridFS file.After the successful write, MongoDB creates two collections in the specified database: fs.chunks and fs.files. These collections can be queried from the mongo shell or from the driver APIs.Reading from GridFS:To read from the GridFS collection, use MongoDB\GridFS\Bucket::downloadToStream function. In this case, an output steam is to be created and the downloadToStream function reads from the stored string to the stream. Then, extract the string from the stream.
Code:uploadFromStream($filename, $source, array $options = [])$filenamefilenamefs.files$source$streamToUploadFromInputStream streamToUploadFrom = new ByteArrayInputStream(inputString.getBytes("UTF-8"));$bucket = (new MongoDB\Client)->test->selectGridFSBucket();
$bucket->uploadFromStream('my-file', $streamToUploadFrom);
_idfs.chunksfs.filesmongoMongoDB\GridFS\Bucket::downloadToStreamdownloadToStream

[User]
Text:Before you answered I already figured this out, thank you anyway for posting this. I’m pretty sure that it will help someone if he searches Google. After solving this problem I wrote a short “tutorial” on how to store and retrieve files from the database, you can find it here.
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Free Certification for Student and tags: ['student-developer-pack']

[User]
Text:I have signed up in MongoDB with my GitHub Student Account. Is it right that I can get free certification if i have signed with student account?
Code:

[Assistant]
Text:Here is a link to the MongoDB Student Pack.Free certification means, I think, free registration to take the certification exam. Normally there is a fee to take the certification exam.
Code:

[Assistant]
Text:Hi @Prasad_Saya, that’s correct Once you’ve completed one of our Learning Paths, you’ll receive 100% discount to the exam.
Code:

[Assistant]
Text:Hello @Lieke_Boonis this discount bound to the student program or does this count in general?Michael
Code:

[Assistant]
Text:Hi Michael,This offer is part of our MongoDB Students pack Best,Lieke
Code:

[Assistant]
Text:It’s written
During this COVID-19 time, we are here to help you. Complete one of our learning paths and enrich your resume with our free certification!So it will be available only for some Time
Code:

[Assistant]
Text:Hi @Alex_BeckhamWelcome to the forum!That’s correct. It might change in the future, but at this moment we offer the free certification temporarily now that most schools & universities are closed. We don’t know when the COVID-19 pandemic will end, but we’re expecting that this situation will last for a while longer, and we’re not planning on changing this offer in the close future.
Code:

[Assistant]
Text:So Which Certificates do we get for freeBoth Exams Listed here MongoDB Courses and Trainings | MongoDB UniversityBoth have price tags of  150.00 USD , this price will be forgiven , Just asking
Code:

[Assistant]
Text:Hi @Alex_Beckham I’m pretty sure that the voucher is only for a single exam. You would have to choose between the DBA or the Developer certification.If my understanding on that is incorrect then hopefully @Lieke_Boon will come by soon and correct me.
Code:

[Assistant]
Text:That’s correct @Doug_Duncan, thank you!Once you’ve completed one of our Learning Paths , you’ll receive 100% discount to the exam. So it’s either for the DBA or for the Developer certification exam.
Code:

[Assistant]
Text:@Lieke_Boon , Please could you explain how to request a 100% discount? I completed the   “Developer Learning Path” using my Github Students pack
Code:

[Assistant]
Text:Hi @Osama_Rashwan,Welcome to the forum! I’ve responded to your email with the code Good luck!Lieke
Code:

[Assistant]
Text:HiTo receive a code, please follow the instructions mentioned here: MongoDB Student Pack.Thank you!
Code:

[Assistant]
Text:I have completed the DBA Path.
When will I get the voucher?
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Locking Documents In Mongo and tags: []

[User]
Text:I would like to know if we can lock the documents in Mongo. We have a .NET service which has 3 different instances of itself. It looks into a collection and gets the document from Mongo.What we want is, if the first instance of the service gets the Document-1 from the collection, then the second instance should not be able to get the same document (Document-1) as it is being processed by another instance. The second instance should get next available document (e.g Document-2). Same with third instance, it should not be able to find the Document-1 or Document-2. It should pick Document-3. But if the first document is not processed correctly, then the lock from that document should be released and the next available service instance should pick it up.Please help. Thank you.
Code:

[Assistant]
Text:Hi @Jason_Widener,In MongoDB we recommend using the findAndModify command for this scenario.This command is atomic and thus lock the document for a status change.Each service instance should do:This way only one  service only will see the pending document.If the processing fails you can change the status back to pending and sort by creation datefor another service to pick it up.If there is a more complex logic consider using transactions.Please let me know if you have any questions.Best regards
Pavel
Code:findAndModifydb.coll.findAndModify({id : "doc1", status : "pending"},{$set : { status : "processing"}});

[User]
Text:Hello Pavel,Thank you so much for your reply. It is really helpful. Just one more question, let say we use findAndModify and  update the status to “Processing” when a service picks up the document. And after some time, the service gets stuck for some time and does not respond anything (not error and also not successful). So in case if the document is in “processing” status for more then 15 min, we want to change the status to “Available” so that next time service runs, document can be picked again. Is it possible to do in Mongo ?Thank you,
Jason
Code:

[Assistant]
Text:Hi @Jason_Widener,We can only offer a TTL partial index on status : “processing” with 15 min time to live however, it will only remove the document.Then you can consider listen to those deletes via a changestream or an Atlas trigger (if this is running on MongoDB Atlas cluster) and recreate the record.However, this might be challenging and not really error proof.I will suggest 2 approaches:Best regards,
Pavel
Code:

[User]
Text:Awesome. thank you so much for your help.
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: DeprecationWarning being triggered even with useUnifiedTopology set to true and tags: ['node-js']

[User]
Text:Hi,The following message is being triggered whenever I run my node backend.
DeprecationWarning: current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.My environment:Connection:
.env.jsThe connection works as well as every database operation, yet I can’t get rid of this warning.Does anyone have any idea why this is happening?Thanks in advance.
Code:MONGO_DB_URL=mongodb://localhost:27017
AUTH_DB_NAME=authentication
const MongoClient = mongodb.MongoClient
const url = process.env.MONGO_DB_URL
const mongodbOptions = {
useNewUrlParser: true,
useUnifiedTopology: true
}
const dbClient = new MongoClient(url, mongodbOptions)

const dbName = process.env.AUTH_DB_NAME

const authenticationDb = async () => {
  const dbClient = await client
  if (!dbClient.isConnected()) {
    await dbClient.connect()
  }
  const db = dbClient.db(dbName)
  return db
}

[Assistant]
Text:Please check these linkshttps://jira.mongodb.org/browse/NODE-2138
Code:

[User]
Text:Thank you, Ramachandra.
Code:

[User]
Text:I’ve gone through all the threads, including the linked one (which I actually had already researched beforehand), but as for the code I posted, I am already doing what it was supposed to be done, according to the threads info.
The useUnifiedTopology: true option is being passed to the MongoClient, not to the client.connect.
The used packages, as well as the MongoDB itself, are up to date. Maybe I’m missing something, but I could not find a solution for this warning in those threads. Would you recommend any step further?
I truly appreciate the help and attention.
Code:

[Assistant]
Text:Hi @Christian_Saboia,
If you are indeed passing useUnifiedTopology: true to your MongoClient constructor, then this message should not display. Can you please update your code sample above if it does not accurately reflect the code you are running?  Specifically, I’m seeing this code which seems to refer to a variable which does not exist:
Code:useUnifiedTopology: trueMongoClientconst dbClient = await client

[User]
Text:Hi Matt,
Actually, there are to files. One is called db.js, which follows:The second one is called authenticationDB.js:This is the actual code, which, despite de warning is working properly.
Thanks in advance!
Code:import mongodb from 'mongodb'
import Grid from 'gridfs-stream'
import multer from 'multer'
import GridFsStorage from 'multer-gridfs-storage'
import path from 'path'
import crypto from 'crypto'

const MongoClient = mongodb.MongoClient
const url = process.env.MONGO_DB_URL
const mongodbOptions = {
  useNewUrlParser: true,
  useUnifiedTopology: true
}

const client = new MongoClient(url, mongodbOptions)

const fileHandler = (bucketName) => (req, file) => {
  return new Promise((resolve, reject) => {
    crypto.randomBytes(16, (err, buf) => {
      if (err) {
        return reject(err)
      }
      const filename = buf.toString('hex') + path.extname(file.originalname)

      // The prop metadata.expires indicates an expiration date in ms for the case where the request fails for any reason.
      // In this (failure) scenario, the expired files can be removed safely.
      // It is the responsability of further request processors (controllers), to remove metadata.expires property
      // from the files it has properly processed or to remove the file (or files), when applicable.
      // An aditional cron job may be created in order to remove orfanned expired files.
      const fileInfo = {
        filename: filename,
        bucketName,
        metadata: {
          originalName: file.originalname,
          encoding: file.encoding,
          mimetype: file.mimetype,
          size: file.size,
          expires: Date.now() + 1000 * 60 * 60
        }
      }
      resolve(fileInfo)
    })
  })
}

const storage = (dbName, bucketName) => {
  const storageUrl = `${url}/${dbName}`
  console.log({ storageUrl })

  return new GridFsStorage({
    url: storageUrl,
    file: fileHandler(bucketName)
  })
}

// accepted options: { fileFilter, limits, preservePath }
const makeUpload = async (dbName, bucketName, options = {}) =>
  multer({ storage: storage(dbName, bucketName), ...options })

const makeGridStream = async (dbName, bucketName) => {
  if (!client.isConnected()) {
    await client.connect()
  }
  const db = client.db(dbName)
  const gfs = Grid(db, mongodb)
  gfs.collection(bucketName)
  return gfs
}

const makeBucket = async (dbName, options) => {
  if (!client.isConnected()) {
    await client.connect()
  }
  const db = client.db(dbName)
  return new mongodb.GridFSBucket(db, options)
}

export { client, makeUpload, makeGridStream, makeBucket }
import { client } from './db'

const dbName = process.env.AUTH_DB_NAME

const authenticationDb = async () => {
  const dbClient = await client
  if (!dbClient.isConnected()) {
    await dbClient.connect()
  }
  const db = dbClient.db(dbName)
  return db
}

export default authenticationDb

[User]
Text:I’ve managed to find out what was happening. The error was on GridFsStorage creation. I’ve added options property and the message is gone.
Thanks.
Code:

[Assistant]
Text:A post was split to a new topic: DeprecationWarning: current Server Discovery and Monitoring engine is deprecate
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Saving a document with large integer values and tags: ['mongoose-odm']

[User]
Text:Heewwoo,I have a problem when I’m creating a new document and saving it to the database, it’s saving the document with wrong data even if the data collected is a specific value set to. I used Mongodb for about 2 years but this is the first time it’s happening to me.
Here is the data that should be stored in the database:

The problem appears on “roleID” which it should be just like in picture, “565193939435388948” (This picture is the data collected, not the data from document inserted.).
And below this is the information that I receive when the document is inserted in database:
As you can see, in the picture, on variable “RoleID”, there is not the same id as in the first picture of the data collected.
(Note: In case anyone says the roleID isn’t string as above, it’s because I took the second picture after tried last time to see if it may work after being converted to number but it doesn’t and just left it like that. being string or number doesn’t change the fact that it’s not saving the data correctly as it was collected.)Practically, from my research, if the number is bigger than 16 characters long, the last 2 characters are like set to 0. I tried to insert a document with 16 characters, it was inserted with the exact data collected. If it’s 17 or 18, the last 2 digits are set to 0.
This is how the code looks like:image812×269 3.8 KBAs well as the Schema form used:
I have used mongodb these 2 years only on windows 7. I have updated recently my pc to windows 10 (not personal choice unfortunately…) and I got this problem. Is because of windows 10?Just to make sure, I’ll say again:
Windows version: Windows 10
Mongodb server version: 4.2.8Can someone guess what is going on? I really like mongodb and I don’t wanna give up on it after I spent so much time learning it.
Code:

[Assistant]
Text:I think it is more related to a javascript,json limitation.Since you get it as a string (like first image), you could change the mongoose schema to a string.  Since it is an id you probably do not make any mathematical computation on it.  A string would be fine.
Code:

[User]
Text:I tried as well to save it as a string but it’s still not working despite chaning mongoose schema to a string.
EDIT: If I try to parse the string and treat it as json just like in the link you sent when adding to database, can this actually help it ?
Edit 1.5: Tried to parse and stringify but no use… Isn’t there a way to like change how json works for mongodb?EDIT 2: I have seen something and Visual studio code gave me a “fix” to make it actually show in database the exact number for the id but that’s working only if I have like the number defined already in a variable and adding an “n” to the end meaning it “covert to bigint numeric literal” but how I can do it with a variable name?
Code:

[Assistant]
Text:My understanding is that it has to be a string all the way.  Otherwise as soon at it is jsonyfied as a number the issue will happen.
Code:

[User]
Text:What do you mean byit has to be a string all the wayI have changed it to be A string when collected data and modify in the schema the type to be a string.
what else I’m missing?EDIT: There isn’t a way to change this in the mongod.cfg file?
Code:

[Assistant]
Text:The file mongod.cfg has nothing to do with that.what else I’m missing?It is really hard to tell from my end.  May be an API in your application is still using it as a number.
Code:

[User]
Text:Can I do something to help you
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Rename Project and Cluster  and tags: []

[User]
Text:Hey, everyone. How’s everyone doing with your projects?I had to rename my project and now I want to know if is possible to rename the cluster on mongodb Atlas. Is it?Thank you all in advance.Daniel Gomes
Code:

[Assistant]
Text:I don’t think Cluster name can be changed once you created it
However project name can be changed
Select the project you want to rename in Atlas.There is a 3 dots button on top left
When you click it you can see project settings.Then edit the project name using pencil icon
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Native query execution in Java and tags: ['java']

[User]
Text:I have native query of mongo db and I want execute the same native query in Java. How can I execute that ?
Code:

[Assistant]
Text:Hello and welcome to MongoDB forum.You can use the Java Driver’s MongoDatabase#runCommand method. This is equivalent to Database Comand - Query and Write Operation Commands in mongo shell
Code:mongo

[Assistant]
Text:Welcome to the community @97vaqasazeem_N_A,Can you provide an example of the query you are trying to run? By “native query”, are you referring to a JavaScript query in the mongo shell?The Java driver provides a full interface for querying MongoDB. To get started, see MongoDB Java Driver Quick Start and the Java Driver Tutorials.Regards,
Stennie
Code:mongo

[User]
Text:Thanks for the reply @Stennie_X .  Following is the sample querydb.collectionABC.aggregate(
[
Code:       {
           $match: { "xxx" : "value" }

       },

       { $sort : { created_date : 1} },

       {
           $group:{
               _id: {"xyz" : "$xyz"}
               _name: {"abc" : "$abc"},
               count: { $sum: 1 }
           }
       },
       {$unwind : "$data"}

    ]
)

[Assistant]
Text:Here is an example with “native” query usage.Sample input documents from a books collection:Note the way you have to extract the result documents - not very covenient. As @Stennie_X has suggested try using the Java Driver provided API.As such you can create the “native” Aggregation pipeline in the MongoDB Compass GUI tool using the Aggregation Pipeline Builder and use the option Export to Specific Language (Java). This generates the Java code, which you can copy and use it in your application - pretty simple.
Code:books{ "title" : "Ulysses", "author" : "James Joyce" }
{ "title" : "War and Peace", "author" : "Leo Tolstoy" }
{ "title" : "Anna Karenina", "author" : "Leo Tolstoy" }

// Aggregation pipeline stages
String match = "{ '$match':{ 'author': 'Leo Tolstoy' } }";
String sort = "{ '$sort':{ 'title': 1} }";

// Build pipeline as a Bson
String pipe = match + ", " + sort;
String strcCmd = "{ 'aggregate': 'books', 'pipeline': [" + pipe + "], 'cursor': { } }";
Document bsonCmd = Document.parse(strCmd);

// Execute the native query
Document result = db.runCommand(bsonCmd);

// Get the output
Document cursor = (Document) result.get("cursor");
List<Document> docs = (List<Document>) cursor.get("firstBatch");
docs.forEach(System.out::println);


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Tunneling (Port forwarding) with MongoDB Go driver is not working and tags: ['golang']

[User]
Text:I am trying to connect aws documentdb from local environment following the docs Connecting to an Amazon DocumentDB Cluster from Outside an Amazon VPC - Amazon DocumentDB
i am able to connect from mongo shell but not from GO applicationTunneling command
ssh -L 27017::27017 @ -NMongo Shell(this is working)
mongo --sslAllowInvalidHostnames --ssl --host localhost:27017 --sslCAFile rds-combined-ca-bundle.pem --username  --password Sample Go codes(This is not working)
connectionURI := mongodb://:@localhost:27017/?sslCAFile=rds-combined-ca-bundle.pem&replicaSet=rs0&readPreference=secondaryPreferred&ssl=true&sslAllowInvalidHostnames=true&retryWrites=false&sslInsecure=true&sslVerifyCertificate=falsetlsConfig, err := getCustomTLSConfig(caFilePath)
if err != nil {
fmt.Printf(“Failed getting TLS configuration: %v”, err)
}// Connect to MongoDB
client, err := mongo.NewClient(options.Client().ApplyURI(connectionURI).SetTLSConfig(tlsConfig))
//client, err := mongo.NewClient(options.Client().ApplyURI(connectionURI))
if err != nil {
fmt.Println("client error: ", err)
}ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()err = client.Connect(ctx)
if err != nil {
fmt.Println("connect error: ", err)
}ctx, cancel = context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()err = client.Ping(ctx, nil)if err != nil {
fmt.Println("ping error: ", err)
}fmt.Println(“Connected to MongoDB!”)collection := client.Database(“mytestdb”).Collection(“mytestcollection”)res, err := collection.InsertOne(context.TODO(), bson.M{“name”: “pi”, “value”: 3.14159})
if err != nil {
fmt.Printf(“Failed to insert document: %v”, err)
}getting following errors
i. ping error:  context deadline exceeded
ii. Failed to insert document: server selection error: server selection timeout, current topology: { Type: ReplicaSetNoPrimary, Servers: [{ Addr: test-1.test.us-west-2.docdb.amazonawscom:27017, Type: Unknown, State: Connected, Average RTT: 0, Last error: connection() : connection(test-docdb-1.test.us-west-2.docdb.amazonawscom:27017[-3]) incomplete read of message header: EOF }, { Addr: test-docdb-3.test.us-west-2.docdb.amazonawscom:27017, Type: Unknown, State: Connected, Average RTT: 0, Last error: connection() : connection(test-docdb-3.test.us-west-2.docdb.amazonawscom:27017[-4]) incomplete read of message header: EOF }, { Addr: test-docdb-2.test.us-west-2.docdb.amazonawscom:27017, Type: Unknown, State: Connected, Average RTT: 0, Last error: connection() : connection(test-docdb-2.test.us-west-2.docdb.amazonawscom:27017[-5]) incomplete read of message header: EOF }, ]I would greatly appreciate for any help/suggestion.(I am new to Go language)
Thanks
Code:

[Assistant]
Text:Hi @Manoj_Maharjan, and welcome to the forum!Please note that AWS DocumentDB API is an emulation of MongoDB which differs in features, compatibility, and implementation from an actual MongoDB deployment. Their suggestion of API version support (eg 3.6) is referring to the wire protocol used rather than the full MongoDB feature set for that version. Official MongoDB drivers (i.e. MongoDB Go driver) are only tested against actual MongoDB deployments.For further questions on AWS DocumentDB connections I’d suggest to contact AWS.Depending on your requirements, you may find it useful to know that MongoDB Atlas clusters can be deployed in AWS, GCP, and Azure. MongoDB Atlas also supports network peering connections for AWS, GCP and Azure-backed clusters.  See also MongoDB Atlas: Set up a Network Peering Connection.
For more information, see Connect to an Atlas clusterRegards,
Wan.
Code:

[Assistant]
Text:I am having the same issue in a similar situation (using AWS DocumentDB). The issue seems to be entirely within the driver before any commands are sent to the server.I tried to trace it back but wasn’t able to follow the driver code very well. What seems to be happening is that the driver appears to recurse through the connection a couple of times to apply changes, and it ends up replacing the “localhost” address with the TLS server name (the DocDB instance) at some point. Since the address of the instance is not Internet-connected, the connection fails and the entire process aborts. The connection must be made through localhost via the SSH tunnel.When I add c.addr = "localhost:27017" to the top of (*topology.connection).connect(...), which forces the update to the address to be undone, I am able to establish a connection. I am not certain whether the behavior described earlier is a bug or not, but for me, it is. And, official tools like MongoDB Compass and the mongo shell do not have this issue.
Code:localhostc.addr = "localhost:27017"(*topology.connection).connect(...)mongo

[User]
Text:Thank you @wan and @Marshall_Meng for your responses. It seems AWS doesn’t allow auto discoverr to all the endpoints. Basically, it allows only one cluster endpoint(primary) connection while tunneling from local environment and  i had to set following 2 parameters to connect directly to the primary endpoint. i am able to connect successfully now.
“&connect=direct&sslInsecure=true”Complete connection uri:
connectionURI := "mongodb://<user_name>:<password>@localhost:27017/?ssl=true&sslCAFile=<path to rds-combined-ca-bundle.pem>&connect=direct&sslInsecure=true&replicaSet=rs0&readPreference=secondaryPreferred
Code:connectionURI := "mongodb://<user_name>:<password>@localhost:27017/?ssl=true&sslCAFile=<path to rds-combined-ca-bundle.pem>&connect=direct&sslInsecure=true&replicaSet=rs0&readPreference=secondaryPreferred

[Assistant]
Text:Thanks for the update Manoj. I can confirm this worked for me as well.
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: How to store user defined schema and save data based on that schema and tags: []

[User]
Text:we want to allow user to define schema and upload data to save based on that schema,
what’s the best way to store the “user define schema” in mongodb, as Text ? or Json?one user can have
var userSchemaB = {
title: String,
body: String
};another user can have
var userSchemaA= {
productName: String,
productDescription: String
};
Code:

[Assistant]
Text:Hello, @Level_0_You!MongoDB has the flexible schema. It means, that you can insert documents with completely different structure in your collection. But in this case, you would need to do the data validation on application side, before each write-operation (create, update).Although, MongoDB supports schema validation, but it will validate all the documents in a collection against the same schema. It means, that you can not have multiple schemas for a single collection.You can, however, use MongoDB schema validation with different schemas by creating a dedicated collection with its own schema per each user:That would work, but you can easily reach limitations collection number if your db instance is provided by some Cloud provider. However, if you have deployed MongoDB on your own servers, the limitations are much more compromising. So, use this at with extreme caution and only if you’re sure it will not create any complications in your application code.Better to think, if you can standardize the data format, coming from your users. This could make your developer’s life much easier: you could use only 1 schema for your single collection and all the validation could be done on database side.If you can not control user’s payload, then consider this 3-step solution:Define schema for each user. You can use shema-inspector shemas for schema definitions and validations, if you have Node.js on backend side. If you use other programming language - search a lib, that can support validation of documents, based on predefined schema.Persist the schemas in some user-related collection. You will need to fetch those schemas each time, when user would want to create/update the product.Validate user’s payload object before inserting or updating document:
Code:const userId = <id>;
db.getCollection(`{userId}_products`).insertOne(<payload>);

[User]
Text:Persist the schemas in some user-related collection. You will need to fetch those schemas each time, when user would want to create/update the product.@slava thanks a lot for the response. after schema1 and schema2 is defined from the UI, how do i save it in the database before i fetch it?  I tried to save it as Object Data type, but it doesnt work. each schema is for a different collection, so user can upload data into each collection based on the defined schema.
Code:

[Assistant]
Text:Please, share:
Code:

[User]
Text:This is the code to save userDefinedSchema in the database, idea is that later i can read it back from tableSetup collection and use it to save new data for the user defined collection.still new to mongodb, not sure whether this is the right approach. thanks for your reply.
Code:			var userDefinedSchema = {
				  title: String,
				  body: String,
				  date:
					{
					  type: String,
					  default: Date.now()
					}
				  };

			var tableUploadSetup = {
			  setupId:String,
			  userDefinedSchemaDetails:Object
			  };

			const tableSetupSchema = new Schema(tableUploadSetup);
			const newTableSetupModel = moogoose.model('TableSetup',tableSetupSchema);

			const newTableSetupData = {
			  setupId: 'SETUP001',
			  userDefinedSchemaDetails:userDefinedSchema
			};

			const newTableSetup = new newTableSetupModel(newTableSetupData);

			newTableSetupModel.save((error) =>{
			if(error){
				console.log('error: '+error)
			  } else{
				console.log('data is saved !!!')
			   }
			 })

[Assistant]
Text:Ok, I must admit, schema management with mongoose is easy, but the schema can not be easily saved to db. Try schema-inspector instead. It’s schema definitions can be safely stored in DB.  Here is an example of how it can be used:
Code:const { createConnection } = require('./helpers');
const { ObjectId } = require('bson');

const inspector = require('schema-inspector');

(async () => {
  // connect to your mongodb somehow
  const url = 'url-to-your-db';
  const connection = await createConnection(url);

  // create links to your collections for later usage
  const dbName = 'for-test';
  const usersColl = connection.db(dbName).collection('users');
  const productsColl = connection.db(dbName).collection('products');

  // get userId somehow
  const userId = new ObjectId();

  // accept user's schema definition
  // from request.body, for example
  const schemaDefinition = {
    type: 'object',
    properties: {
      productName: {
        type: 'string',
        optional: false,
        minLength: 2
      },
      productDescription:  {
        optional: true
      },
    }
  };

  // write that definition for later usage
  await usersColl.insertOne({
    _id: userId,
    productSchema: schemaDefinition
  });

  // when user wants to create / update document,
  // fetch the definitions from the collection
  const user = await usersColl.findOne({ _id: userId });

  // get create / update payload somehow
  const payload = {
    productName: 'B',
  };

  // validate payload against your schema
  const validationResult = inspector.validate(user.productSchema, payload)

  if (!validationResult.valid) {
    console.log('errors:', validationResult.error);
    // return errors back to user
  }
  else {
    // insert or update the payload to your db
    productsColl.insertOne(payload);
  }

  console.log('done!');
  process.exit(0);

})();

[User]
Text:@slava thank you so much so much,  i will try it out.
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: How to avoid "duplicate id" error during restoring collections using mongorestore and tags: []

[User]
Text:When restoring collections on destination database mongorestore is giving “duplicate id” error.
Should we treat this as serious Error or just ignore it as Warning or Informational message??
Is it due to the fact that the collections already exist.
In what other cases this error will creep up.
Code:

[Assistant]
Text:Hello @venkata_reddy, welcome to the MongoDB Community. I will explain the “error” with an example.Lets take an example collection movie with two documents:Do a mongodump of this collection and restore it to a new collection movie_new.Now, the movie_new has two documents with _id values 1 and 2.Insert one more document into the movie collection.{ _id: 9, title: "the empire strikes back" }Again, do a mongodump and mongorestore from movie to movie_new. You will see the mongorestore completes with inserting only the new document with _id: 9 into the movie_new collection.During the restore process you will see messages on the console like this:In the above output, the error 2020-07-03T07:38:08.627+0530  continuing through error: E11000 duplicate key error collection: test.movie_new index: _id_ dup key: { _id: 1 } is expected, as there is already a document in the target collection with the same _id that is being restored; and the document doesn’t get inserted (see below note on inserts). The restore process continues to process remaining document(s) after logging this “error” message - it does not abort the process.The process completes inserting only one document into movie_new collection.From the MongoDB documentation: mongorestore Inserts Onlymongorestore can create a new database or add data to an existing database. However, mongorestore performs inserts only and does not perform updates. That is, if restoring documents to an existing database and collection and existing documents have the same value _id field as the to-be-restored documents, mongorestore will not overwrite those documents.
Code:movie{ _id: 1, title: "star wars" }
{ _id: 2, title: "return of jedi" }
mongodumpmovie_newos > mongodump  --db=test --collection=movie
os > mongorestore --db=test --collection=movie_new dump/test/movie.bson
movie_new_id12movie{ _id: 9, title: "the empire strikes back" }mongodumpmongorestoremoviemovie_newmongorestore_id: 9movie_new2020-07-03T07:38:08.519+0530    checking for collection data in dump\test\movie.bson
2020-07-03T07:38:08.527+0530    restoring to existing collection test.movie_new without dropping
2020-07-03T07:38:08.532+0530    reading metadata for test.movie_new from dump\test\movie.metadata.json
2020-07-03T07:38:08.537+0530    restoring test.movie_new from dump\test\movie.bson
2020-07-03T07:38:08.627+0530    continuing through error: E11000 duplicate key error collection: test.movie_new index: _id_
 dup key: { _id: 1 }
2020-07-03T07:38:08.631+0530    continuing through error: E11000 duplicate key error collection: test.movie_new index: _id_
 dup key: { _id: 2 }
2020-07-03T07:38:08.638+0530    restoring indexes for collection test.movie_new from metadata
2020-07-03T07:38:08.646+0530    finished restoring test.movie_new (1 document, 2 failures)
2020-07-03T07:38:08.648+0530    1 document(s) restored successfully. 2 document(s) failed to restore.
2020-07-03T07:38:08.627+0530  continuing through error: E11000 duplicate key error collection: test.movie_new index: _id_ dup key: { _id: 1 }_idmovie_newmongorestoremongorestore_idmongorestore

[User]
Text:Hi @Prasad_Saya, Thank you for the useful information. This has clarified my confusion.
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Finding duplicate documents before creating unique index and tags: ['indexes']

[User]
Text:I have added unique key for name field after that I have drop the indexes and I am trying to create indexes I am getting duplicate error because we have duplicate documents in the collection. how to get the duplicate documents please help me to resolve this issue
Code:

[Assistant]
Text:Welcome to the community, @Vinay_reddy_Mamedi!Let’s assume we have this data in collection ‘test1’:Then, to find duplicates we can use this aggregation:This will output ids:It is also possible join full documents with duplicated values, if just ids is not enough for you.
You can do this by adding $lookup stage in the end of the pipeline:Output, after adding the $lookup stage:
Code:db.test1.insertMany([
  { _id: 1, val: 'A', },
  { _id: 2, val: 'B', },
  { _id: 3, val: 'C', },
  { _id: 4, val: 'A', },
])
db.test1.aggregate([
  {
    $group: {
      // collect ids of the documents, that have same value
      // for a given key ('val' prop in this case)
      _id: '$val',
      ids: {
        $push: '$_id'
      },
      // count N of duplications per key
      totalIds: {
        $sum: 1,
      }
    }
  },
  {
    $match: {
      // match only documents with duplicated value in a key
      totalIds: {
        $gt: 1,
      },
    },
  },
  {
    $project: {
      _id: false,
      documentsThatHaveDuplicatedValue: '$ids',
    }
  },
]);
{ "documentsThatHaveDuplicatedValue" : [ 1, 4 ] }
{
  $lookup: {
    // note, you need to use same collection name here
    from: 'test1',
    localField: 'documentsThatHaveDuplicatedValue',
    foreignField: '_id',
    as: 'documentsThatHaveDuplicatedValue'
  }
}
{
  "documentsThatHaveDuplicatedValue": [
    {
      "_id" : 1,
      "val" : "A"
    },
    {
      "_id" : 4,
      "val" : "A"
    }
  ]
}


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Nextjs example. Why use next-connect and tags: ['serverless', 'next-js', 'developer-hub']

[User]
Text:Here is an example for how to use MongoDB with nextjs: Building Modern Applications with Next.js and MongoDB | MongoDBWhy is next-connect used here? Why share a connection with middleware? Couldn’t a module be used instead?
Code:

[Assistant]
Text:Hi Roeland, you could. I’m actually making some updates to this article (will be out early next week). There’s been some pretty nice changes to Next.js since this article came out and there are a few improvements I can think of that would make it much better.
Code:

[Assistant]
Text:Hi Roeland - after looking into this some more, I think I have a more elegant solution (I talked to one of the folks over at Vercel, and they recommended this approach). It’s essentially what you suggested, no need to use middleware. You can simply update your database.js to look like this:and then in your API routes, you’d simply import the connectToDatabase method and go from there…Let me know if this helps!
Code:import { MongoClient } from "mongodb";

let uri = "YOUR-CONNECTION-STRING";
let cachedDb = null;

export async function connectToDatabase() {
  if (cachedDb) {
    return cachedDb;
  }
  const client = await MongoClient.connect(uri, { useNewUrlParser: true });
  const db = await client.db("DB-NAME");

  cachedDb = db;
  return db;
}
handler.post(async (req, res) => {
  let data = req.body;
  data = JSON.parse(data);
  data.date = new Date(data.date);
  let db = await connectToDatabase();
  let doc = db
    .collection("daily")
    .updateOne({ date: new Date(data.date) }, { $set: data }, { upsert: true });

  res.json({ message: "ok" });
});

[User]
Text:Yes, I did something similar.
But how does MongoDB handle connections?
The connection is not closed here and since an api route will be deployed to aws lambda, there could be many open connections.
Could that cause problems?
Code:

[User]
Text:I found more about aws lambda and mongodb here:Thanks for the help!
Code:


[System] You are an assistant of MongoDB Community who is responsible for answering the questions related with MongoDB.

[System] The following conversation has title: Does mongodump, dump all the data+indexes of the collection? and tags: []

[User]
Text:iam using mongodump,mongorestore.i would like to know what kind of details from the collection are available in dump exported by mongodump command.
does it export all the index created for specific fields in a collection?
Code:

[Assistant]
Text:does it export all the index created for specific fields in a collection?Yes, mongodump does export the indexes created on the collection, and the indexes are restored with mongorestore along with the data. This is true with MongoDB v4.2.
Code:mongodumpmongorestore

[User]
Text:@Prasad_Saya i don’t think it works like that for me. Below is the problem i’m facing.Collection stats before export
image1734×1102 98.6 KB
i exported this collection using mongodump. below 2 files are generated
Now i used mongorestore to import the file into another mongodb instance
mongorestore --db university --collection users /Users/dev/Documents/mongodb-performance-inputfiles/users.bson.gz --drop --gzipAfter import i don’t see any indexes in stats.
i haven’t done any other steps to import users.metadata.jsonimage1430×770 62.4 KBwhat am i missing here?
should i import users.metadata.json manually?But i read in documentation like , we don’t need to import users.metadata.json manually
my mongorestore logs shows no index to restore, but i have many index to restore as it is shown in my  screenshots
image2430×370 269 KB
Code:mongorestore --db university --collection users /Users/dev/Documents/mongodb-performance-inputfiles/users.bson.gz --drop --gzipusers.metadata.jsonusers.metadata.jsonusers.metadata.json

[Assistant]
Text:I have a collection called as movie with two documents and two indexes: the default index on _id and an index on a field year. I verified the data and indexes using the following methods from mongo shell:From the Windows OS command prompt:os > mongodump  --db=test --collection=movieThis created a folder dump\test with two files: movie.bson, and movie.metadata. The folder and files are created in the directory from where I ran the mongodump command.Again, from the OS command prompt:os > mongorestore --db=test --collection=movie_new dump/test/movie.bsonFrom the shell, I could verify:
Code:movie_idyearmongoshell > db.movie.find()
shell > db.movie.getIndexes()
os > mongodump  --db=test --collection=moviedump\testmovie.bsonmovie.metadatamongodumpos > mongorestore --db=test --collection=movie_new dump/test/movie.bsonshell > db.movie_new.find()
shell > db.movie_new.getIndexes()

[User]
Text:i have to export with gzip option as the collection size is too large. i think with gzip option this doesn’t work.
Code:

[Assistant]
Text:I tried with --gzip option. It worked in my environment correctly.I could verify the restored movie3 collection and its indexes.
Code:--gzip> mongodump  --gzip --db=test --collection=movie
> mongorestore --gzip --db=test --collection=movie3 dump/test/movie.bson.gz
movie3

[Assistant]
Text:@Divine_CutlerIf your dump was gzip the metadata should be gzip too, was it uncompressed accidentally?I can replicate no Indexes to restore if I uncompress the metadata. When I gzipped it back up mongorestore created the indexes.
Code:no Indexes to restore