How To Store MongoDB Documents In A Slice And Iterate Over Them In Golang

Written by Data Pilot

July 17, 2019

BSON
Golang
MongoDB
MongoDB Driver

Introduction to creating MongoDB queries and filters in Golang

If you’re working with MongoDB and Golang, it’s important to know how to create and format MongoDB filter queries so that they can be passed into method calls. Fortunately, this process is fairly simple and straightforward. In this article, we’ll show you how to create BSON map objects from raw JSON strings and from primitive.M BSON objects. We’ll also explain how to pass these objects to the Golang driver’s method calls to query MongoDB documents.

Prerequisites for making API calls to MongoDB using the official Golang driver

Before we dive into the Golang code and learn how to make API calls to MongoDB, there are a few prerequisites that need to be reviewed:

MongoDB needs to be installed and running on the same server that’s making the Go API calls. You can type mongod in your terminal window to get information on the MongoDB daemon processes.
You’ll need to have a few documents in a MongoDB collection that you can query in your Go script:

Screenshot of documents in a collection on the MongoDB Compass UI

If you’d like to access the attributes and fields of a Golang object, you’ll need to use a third-party library called oleiade/reflections, which returns high-level abstractions of Go’s native reflect library. To install this package, enter the following commands in your terminal or command prompt window:

1	go get gopkg.in/oleiade/reflections.v1

Golang also needs to be installed, and the MongoDB project directory needs to be in Go’s $GOPATH:

1	go get go.mongodb.org/mongo-driver/mongo

You can get information on the current value of GOPATH by using the commands go help gopath or $GOPATH. For more help with Go’s get command, enter go help get into a terminal window.

Import the MongoDB and Go packages in a Golang script

Now that we’ve gone over the prerequisites, we’re ready to turn to the code. We’ll start our script by importing the packages for MongoDB that can create BSON filters and queries and unmarshal strings in the main() function:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

Declare a new struct for the MongoDB document fields

We’ll use Golang’s struct datatype to map the MongoDB fields for the collection’s documents:

1
2
3
4
5
6
7
8
9
10
11

type MongoFields struct {
Key string `json:"key,omitempty"`
// ObjectId() or objectid.ObjectID is deprecated--use primitive instead
ID primitive.ObjectID `bson:"_id, omitempty"`

// Use these field tags so Golang knows how to map MongoDB fields
// `bson:"string field" json:"string field"`
StringField string `bson:"string field" json:"string field"`
IntField int `bson:"int field" json:"int field"`
BoolField bool `bson:"bool field" json:"bool field"`
}

View a collection’s document fields in the Mongo Shell

If you’re not sure of the datatypes for a collection’s document fields, simply enter these commands in the Mongo Shell to view the exact format of the fields:

1
2
3
4

mongo shell
use SomeDatabase
db["Some Collection"]
db["Some Collection"].findOne()

Screenshot of mongo Shell in a terminal getting a document in a MongoDB collection

Declare the Golang main() function and connect to MongoDB

Now we’ll declare the main() function, which is necessary for the MongoDB API calls and document iteration. We can connect to the MongoDB server using mongo.Connect() and have it return a client instance that will be used to get a collection’s documents:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

func main() {

// Declare host and port options to pass to the Connect() method
clientOptions := options.Client().ApplyURI("mongodb://localhost:27017")

// Connect to the MongoDB and return Client instance
client, err := mongo.Connect(context.TODO(), clientOptions)
if err != nil {
fmt.Println("mongo.Connect() ERROR:", err)
log.Fatal(err)
}

// Declare Context object for the MongoDB API calls
ctx := context.Background()

// Access a MongoDB collection through a database
col := client.Database("SomeDatabase").Collection("Some Collection")

When you’re adding this code to your script, be sure to replace the "SomeDatabase" and "Some Collection" strings shown in the example with the correct names for your MongoDB database and collection.

Declaring a BSON map MongoDB filter object

Next, we’ll nest a BSON query into another BSON map object which has one of the collection’s fields as its key:

1	filter := bson.M{"int field": bson.M{"$gt":42}}

Build a MongoDB filter query from a string in Golang

In this section, we’ll look at the process of creating a BSON map object from a string to use for a MongoDB query in Golang.

Declare a Go string containing a MongoDB JSON filter

We’ll use backticks to declare a raw string of a valid JSON object in Golang; this MongoDB JSON string in Golang will be used for the filter. Then, we’ll instantiate an empty bson.M object for the unmarshaled data to be stored in:

1
2
3
4
5

// Create a string using ` string escape ticks
query := `{"$eq":"last value"}`

// Declare an empty BSON Map object
var bsonMap bson.M

Unmarshal a Go string using the JSON package

After that, we’ll unmarshal our string and return a primitive.M object:

1
2
3
4
5
6
7
8
9

// Use the JSON package's Unmarshal() method
err = json.Unmarshal([]byte(query), &bsonMap)
if err != nil {
log.Fatal("json.Unmarshal() ERROR:", err)
} else {
fmt.Println("bsonMap:", bsonMap)
fmt.Println("bsonMap TYPE:", reflect.TypeOf(bsonMap))
fmt.Println("BSON:", reflect.TypeOf(bson.M{"int field": bson.M{"$gt":42}}))
}

Declare a nested bson.M MongoDB filter from the primitive.M object returned by json.Unmarshal()

1 2	// Nest the Unmarshalled BSON map inside another BSON object filter := bson.M{"string field": bsonMap}

Use the MongoDB filter to return a Cursor object and iterate over the documents

At this point, we’ll have the Golang driver’s Find() method return a Cursor object. Then we’ll iterate over the documents by passing a context object to the cursor’s Next() method:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

// Pass the filter to Find() to return a MongoDB cursor
cursor, err := col.Find(ctx, filter)
if err != nil {
log.Fatal("col.Find ERROR:", err)
}

// Print cursor object
fmt.Println("\ncursor TYPE:", reflect.TypeOf(cursor))
fmt.Println("cursor:", cursor)

// iterate through all documents
for cursor.Next(ctx) {
var p MongoFields

// Decode the document
if err := cursor.Decode(&p); err != nil {
log.Fatal("cursor.Decode ERROR:", err)
}
// Print the results of the iterated cursor object
fmt.Printf("\nMongoFields: %+v\n", p)
}
}

Conclusion

Being able to construct filter queries is essential if you want to harness the power of MongoDB in your Golang scripts. Using strings and BSON map objects, it’s easy to work with MongoDB filters in Golang. With the help of the examples provided in this tutorial, you’ll have a better understanding of how to create a MongoDB filter query using BSON in Golang.

Just the Code

We’ve looked at our code one section at a time so far. Shown below is the entire script used to query for MongoDB documents with BSON in Golang:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90

package main

import (
"context" // manage multiple requests
"encoding/json" // Use JSON encoding for bson.M string
"fmt" // Println() function
"log"
"reflect" // get an object type

// import 'mongo-go-driver' package libraries
"go.mongodb.org/mongo-driver/bson"
"go.mongodb.org/mongo-driver/bson/primitive" // for BSON ObjectID
"go.mongodb.org/mongo-driver/mongo"
"go.mongodb.org/mongo-driver/mongo/options"
)

type MongoFields struct {
Key string `json:"key,omitempty"`
// ObjectId() or objectid.ObjectID is deprecated--use primitive instead
ID primitive.ObjectID `bson:"_id, omitempty"`

// Use these field tags so Golang knows how to map MongoDB fields
// `bson:"string field" json:"string field"`
StringField string `bson:"string field" json:"string field"`
IntField int `bson:"int field" json:"int field"`
BoolField bool `bson:"bool field" json:"bool field"`
}

func main() {

// Declare host and port options to pass to the Connect() method
clientOptions := options.Client().ApplyURI("mongodb://localhost:27017")

// Connect to the MongoDB and return Client instance
client, err := mongo.Connect(context.TODO(), clientOptions)
if err != nil {
fmt.Println("mongo.Connect() ERROR:", err)
log.Fatal(err)
}

// Declare Context object for the MongoDB API calls
ctx := context.Background()

// Access a MongoDB collection through a database
col := client.Database("SomeDatabase").Collection("Some Collection")

// Typical BSON Map filter
//filter := bson.M{"int field": bson.M{"$gt":42}}

// Create a string using ` string escape ticks
query := `{"$eq":"last value"}`

// Declare an empty BSON Map object
var bsonMap bson.M

// Use the JSON package's Unmarshal() method
err = json.Unmarshal([]byte(query), &bsonMap)
if err != nil {
log.Fatal("json.Unmarshal() ERROR:", err)
} else {
fmt.Println("bsonMap:", bsonMap)
fmt.Println("bsonMap TYPE:", reflect.TypeOf(bsonMap))
fmt.Println("BSON:", reflect.TypeOf(bson.M{"int field": bson.M{"$gt": 42}}))
}

// Nest the Unmarshalled BSON map inside another BSON object
filter := bson.M{"string field": bsonMap}

// Pass the filter to Find() to return a MongoDB cursor
cursor, err := col.Find(ctx, filter)
if err != nil {
log.Fatal("col.Find ERROR:", err)
}

// Print cursor object
fmt.Println("\ncursor TYPE:", reflect.TypeOf(cursor))
fmt.Println("cursor:", cursor)

// iterate through all documents
for cursor.Next(ctx) {
var p MongoFields

// Decode the document
if err := cursor.Decode(&p); err != nil {
log.Fatal("cursor.Decode ERROR:", err)
}
// Print the results of the iterated cursor object
fmt.Printf("\nMongoFields: %+v\n", p)
}
}

Pilot the ObjectRocket Platform Free!

Try Fully-Managed CockroachDB, Elasticsearch, MongoDB, PostgreSQL (Beta) or Redis.

Get Started