Golang Tutorial: Unit Testing with Mocking

Unit testing has always been my thing, almost like a hobby. There was a time when I was obsessed with it, and I made sure that all my projects had at least 90% unit test coverage. You can probably imagine how much time it can take to make such a significant change in the codebase. However, the result was worth it because I rarely encountered bugs related to business logic. Most of the issues were related to integration problems with other services or databases.

Adding new business rules was a breeze because there were already tests in place to cover all the cases from before. The key was to ensure that these tests remained successful in the end. Sometimes, I didn’t even need to check the entire running service; having the new and old unit tests pass was sufficient.

Once, while working on a personal project, I had to write unit tests to cover numerous Go structs and functions—more than 100 in total. It consumed my entire weekend, and late on a Sunday night, before heading out on a business trip the next day, I set an alarm clock to wake me up. I had hardly slept that night; it was one of those restless nights when you dream but are also aware of yourself and your surroundings. My brain was active the entire time, and in my dreams, I kept writing unit tests for my alarm clock. To my surprise, each time I executed a unit test in my dream, the alarm rang. It continued ringing throughout the night.

And yes, I almost forgot to mention, for two years, we had zero bugs in production. The application continued to fetch all the data and send all the emails every Monday. I don’t even remember my Gitlab password anymore.

Unit Testing and Mocking (in general)

In Martin Fowler’s article, we can identify two types of unit tests:

1. Sociable unit tests, where we test a unit while it relies on other objects in conjunction with it. For example, if we want to test the UserController, we would test it along with the UserRepository, which communicates with the database.

2. Solitary unit tests, where we test a unit in complete isolation. In this scenario, we would test the UserController, which interacts with a controlled, mocked UserRepository. With mocking, we can specify how it behaves without involving a database.

Both approaches are valid and have their place in a project, and I personally use both of them. When writing sociable unit tests, the process is straightforward; I utilize the components already present in my module and test their logic together. However, when it comes to mocking in Go, it’s not a standard procedure. Go doesn’t support inheritance but relies on composition. This means one struct doesn’t extend another but contains it. Consequently, Go doesn’t support polymorphism at the struct level but instead relies on interfaces. So, when your struct depends directly on another struct instance or when a function expects a specific struct as an argument, mocking that struct can be challenging.

In the code example below, we have a simple case with UserDBRepository and AdminController. AdminController directly depends on an instance of UserDBRepository, which is the implementation of the Repository responsible for communicating with the database.

UserDBRepository struct

type UserDBRepository struct {
	connection *sql.DB
}

func (r *UserDBRepository) FilterByLastname(ctx context.Context, lastname string) ([]User, error) {
	var users []User
	//
	// do something with users
	//
	return users, nil
}

AdminController struct

type AdminController struct {
	repository *UserDBRepository
}

func (c *AdminController) FilterByLastname(ctx *gin.Context) {
	lastname := ctx.Param("name")
	c.repository.FilterByLastname(ctx, lastname)
	//
	// do something with users
	//
}

When it comes to writing unit tests for the AdminController to check if it generates the correct JSON response, we have two options:

1. Provide a fresh instance of UserDBRepository along with a database connection to AdminController and hope that it will be the only dependency you need to pass over time.

2. Don’t provide anything and expect a nil pointer exception as soon as you start running the test.

To avoid the latter case and to enable proper unit testing, we need to ensure that our code adheres to the Dependency Inversion Principle. Once we this principle, our refactored code takes on the shape shown in the example below. In this improved structure, the actual AdminController depends on the UserRepository interface, without specifying whether it’s a repository for a database or something else.

UserRepository interface

type UserRepository interface {
	GetByID(ctx context.Context, ID string) (*User, error)
	GetByEmail(ctx context.Context, email string) (*User, error)
	FilterByLastname(ctx context.Context, lastname string) ([]User, error)
	Create(ctx context.Context, user User) (*User, error)
	Update(ctx context.Context, user User) (*User, error)
	Delete(ctx context.Context, user User) (*User, error)
}

AdminController struct

type AdminController struct {
	repository UserRepository
}

func NewAdminController(repository UserRepository) *AdminController {
	return &AdminController{
		repository: repository,
	}
}

func (c *AdminController) FilterByLastname(ctx *gin.Context) {
	lastname := ctx.Param("name")
	c.repository.FilterByLastname(ctx, lastname)
	//
	// do something with users
	//
}

Now that we have a starting point, let’s explore how we can perform mocking most effectively.

Generate Mocks

There are several libraries for generating mocks, and you can even create your own generator if you prefer. Personally, I like the Mockery package. It provides mocks that are supported by the Testify package, which is a good enough reason to stick with it.

Let’s revisit the previous example with UserRepository and AdminController. AdminController expects the UserRepository interface to filter Users by their Lastname whenever somebody sends a request to the /users endpoint. Strictly speaking, AdminController doesn’t care about how the UserRepository finds the result. Depending on whether it receives a slice of Users or an error, the crucial part is to attach the appropriate response to the Context from the Gin package.

Application code

func main() {
	var repository UserRepository
	//
	// initialize repository
	//
	controller := NewAdminController(repository)

	router := gin.Default()

	router.GET("/users/:lastname", controller.FilterByLastname)
	//
	// do something with router
	//
}

In this example, I have used the Gin package for routing, but it doesn’t matter which package we want to use for that purpose. We would first initialize the actual implementation of UserRepository, pass it to AdminController, and define endpoints before running our server.

At this point, our project structure may look like this:

Project structure

user-service
├── cmd
│   └── main.go
└── pkg
    └── user
        ├── user.go
        ├── admin_controller.go
        └── admin_controller_test.go

Now, inside the user folder, we can execute the Mockery command to generate mock objects.

Mockery command

$ mockery --all --case=underscore

The content of the generated file looks like the example below:

Generated file

// Code generated by mockery v1.0.0. DO NOT EDIT.

package mocks

import (
	//
	// some imports
	//
	mock "github.com/stretchr/testify/mock"
)

// UserRepository is an autogenerated mock type for the UserRepository type
type UserRepository struct {
	mock.Mock
}

// Create provides a mock function with given fields: ctx, _a1
func (_m *UserRepository) Create(ctx context.Context, _a1 user.User) (*user.User, error) {
	ret := _m.Called(ctx, _a1)

	var r0 *user.User
	if rf, ok := ret.Get(0).(func(context.Context, user.User) *user.User); ok {
		r0 = rf(ctx, _a1)
	} else {
		if ret.Get(0) != nil {
			r0 = ret.Get(0).(*user.User)
		}
	}

	var r1 error
	if rf, ok := ret.Get(1).(func(context.Context, user.User) error); ok {
		r1 = rf(ctx, _a1)
	} else {
		r1 = ret.Error(1)
	}

	return r0, r1
}

// and so on....

When I work on a project, I like to have all commands written somewhere inside the project. Sometimes, it can be a Makefile or a bash script. But here, we can add an additional generate.go file inside the user folder and place the following code inside of it:

File /pkg/user/generate.go

package user

//go:generate go run github.com/vektra/mockery/cmd/mockery -all -case=underscore

New Project structure

user-service
├── cmd
│   └── main.go
└── pkg
    └── user
        ├── mocks
        │   └── user_repository.go
        ├── user.go
        ├── admin_controller.go
        ├── admin_controller_test.go
        └── generate.go

This file contains a specific comment, starting with go:generate. It includes a flag for executing the code after it, and as soon as you run the command from below inside the project’s root folder, it will generate all files:

Generate command

$ go generate ./...

Both approaches ultimately yield the same result — a generated file with a mocked object. So, writing solitary unit tests should no longer be an issue:

File /pkg/user/admin_controller_test.go

func TestAdminController(t *testing.T) {
	var ctx *gin.Context
	//
	// setup context
	//
	
	repository := &mocks.UserRepository{}
	repository.
		On("FilterByLastname", ctx, "some last name").
		Return(nil, errors.New("some  error")).
		Once()
	
	controller := NewAdminController(repository)
	controller.FilterByLastname(ctx)
	//
	// do some checking for ctx
	//
}

Partial mocking of Interface

Sometimes, there is no need to mock all the methods from the interface, or the package is not under our control, preventing us from generating files. It also doesn’t make sense to create and maintain files in our library. However, there are instances when an interface contains numerous methods, and we only need a subset of them. In such cases, we can use an example with UserRepository. AdminController utilizes only one function from the Repository, which is FilterByLastname. This means we don’t require any other methods to test AdminController. To address this, let’s create a struct called MockedUserRepository, as shown in the example below:

MockedUserRepository struct

type MockedUserRepository struct {
	UserRepository
	filterByLastnameFunc func(ctx context.Context, lastname string) ([]User, error)
}

func (r *MockedUserRepository) FilterByLastname(ctx context.Context, lastname string) ([]User, error) {
	return r.filterByLastnameFunc(ctx, lastname)
}

MockedUserRepository implements the UserRepository interface. We ensured this by embedding the UserRepository interface inside MockedUserRepository. Our mock object expects to contain an instance of the UserRepository interface within it. If that instance is not defined, it will default to nil. Additionally, it has one field, which is a function type with the same signature as FilterByLastname. The FilterByLastname method is attached to the mocked struct, and it simply forwards calls to this private field. Now, if we rewrite our test as follows, it may appear more intuitive:

File /pkg/user/admin_controller_test.go

func TestAdminController(t *testing.T) {
	var gCtx *gin.Context
	//
	// setup context
	//

	repository := &MockedUserRepository{}
	repository.filterByLastnameFunc = func(ctx context.Context, lastname string) ([]User, error) {
		if ctx != gCtx {
			t.Error("expected other context")
		}
		
		if lastname != "some last name" {
			t.Error("expected other lastname")
		}
		return nil, errors.New("error")
	}

	controller := NewAdminController(repository)
	controller.FilterByLastname(gCtx)
	//
	// do some checking for ctx
	//
}

This technique can be beneficial when testing our code’s integration with AWS services, such as SQS, using the AWS SDK. In this case, our SQSReceiver depends on the SQSAPI interface, which has many functions:

SQSReceiver

import (
	//
	// some imports
	//
	"github.com/aws/aws-sdk-go/service/sqs/sqsiface"
)

type SQSReceiver struct {
	sqsAPI sqsiface.SQSAPI
}

func (r *SQSReceiver) Run() {
	//
	// wait for SQS message
	//
}

Here we can use the same technique and provide our own mocked struct:

MockedSQSAPI

type MockedSQSAPI struct {
	sqsiface.SQSAPI
	sendMessageFunc func(input *sqs.SendMessageInput) (*sqs.SendMessageOutput, error)
}

func (m *MockedSQSAPI) SendMessage(input *sqs.SendMessageInput) (*sqs.SendMessageOutput, error) {
	return m.sendMessageFunc(input)
}

Test SQSReceiver

func TestSQSReceiver(t *testing.T) {
	//
	// setup context
	//

	sqsAPI := &MockedSQSAPI{}
	sqsAPI.sendMessageFunc = func(input *sqs.SendMessageInput) (*sqs.SendMessageOutput, error) {
		if input.MessageBody == nil || *input.MessageBody != "content" {
			t.Error("expected other message")
		}

		return nil, errors.New("error")
	}

	receiver := &SQSReceiver{
		sqsAPI: sqsAPI,
	}

	receiver.Run()
	//
	// do some checking for ctx
	//
}

In general, I don’t usually test infrastructural objects responsible for establishing connections with databases or external services. For such cases, I prefer to write tests at a higher level of the testing pyramid. However, if there is a genuine need to test such code, this approach has been helpful to me.

Mocking of Function

In core Go code or within other packages, there are many useful functions available. We can use these functions directly in our code, as demonstrated in the ConfigurationRepository below. This struct is responsible for reading the config.yml file and returning the configuration used throughout the application. ConfigurationRepository calls the ReadFile method from the core Go package OS:

Usage of method ReadFile

type ConfigurationRepository struct {
	//
	// some fields	
	//
}

func (r *ConfigurationRepository) GetConfiguration() (map[string]string, error) {
	config := map[string]string{}
	data, err := os.ReadFile("config.yml")
	//
	// do something with data
	//
	return config, nil
}

In code like this, when we want to test GetConfiguration, it becomes necessary to depend on the presence of the config.yml file for each test execution. This means relying on technical details, such as reading from files. In such situations, I have used two different approaches in the past to provide unit tests for this code.

Variation 1: Simple Type Aliasing

The first approach is to create a type alias for the method type that we want to mock. This new type represents the function signature we want to use in our code. In this case, ConfigurationRepository should depend on this new type, FileReaderFunc, instead of the method we want to mock:

Use FileReaderFunc

type FileReaderFunc func(filename string) ([]byte, error)

type ConfigurationRepository struct {
	fileReaderFunc FileReaderFunc
	//
	// some fields
	//
}

func NewConfigurationRepository(fileReaderFunc FileReaderFunc) ConfigurationRepository{
	return ConfigurationRepository{
		fileReaderFunc: fileReaderFunc,
	}
}

func (r *ConfigurationRepository) GetConfiguration() (map[string]string, error) {
	config := map[string]string{}
	data, err := r.fileReaderFunc("config.yml")
	//
	// do something with data
	//
	return config, nil
}

In this case, when initializing our application, we would pass the actual method from the Go core package as an argument during the creation of ConfigurationRepository:

Main function

package main

func main() {
	repository := NewConfigurationRepository(ioutil.ReadFile)
  
	config, err := repository.GetConfiguration()
	//
	// do something with configuration
	//
}

Finally, we can write a unit test as shown in the code example below. Here, we define a new FileReaderFunc function that returns the result we control in each of the cases.

Test with FileReaderFunc

func TestGetConfiguration(t *testing.T) {
	var readerFunc FileReaderFunc

	// we want to have error from reader
	readerFunc = func(filename string) ([]byte, error) {
		return nil, errors.New("error")
	}

	repository := NewConfigurationRepository(readerFunc)
	_, err := repository.GetConfiguration()
	if err == nil {
		t.Error("error is expected")
	}

	// we want to have concrete result from reader
	readerFunc = func(filename string) ([]byte, error) {
		return []byte("content"), nil
	}

	repository = NewConfigurationRepository(readerFunc)
	_, err = repository.GetConfiguration()
	if err != nil {
		t.Error("error is not expected")
	}
	//
	// do something with config
	//
}

Variation 2: Complex Type Aliasing with Interface

The second variation employs the same concept but relies on an interface as a dependency in ConfigurationRepository. Instead of depending on a function type, it depends on an interface FileReader, which has a method with the same signature as the ReadFile method we want to mock.

Use FileReader interface

type FileReader interface {
	ReadFile(filename string) ([]byte, error)
}

type ConfigurationRepository struct {
	fileReader FileReader
	//
	// some fields
	//
}

func NewConfigurationRepository(fileReader FileReader) *ConfigurationRepository {
	return &ConfigurationRepository{
		fileReader: fileReader,
	}
}

func (r *ConfigurationRepository) GetConfiguration() (map[string]string, error) {
	config := map[string]string{}
	data, err := r.fileReader.ReadFile("config.yml")
	//
	// do something with data
	//
	return config, nil
}

At this point, we should once again create the same type alias, FileReaderFunc, but this time we should attach a function to that type. Yes, we need to add a method to a method (I cannot express how much I appreciate this aspect in Go).

New FileReaderFunc

type FileReaderFunc func(filename string) ([]byte, error)

func (f FileReaderFunc) ReadFile(filename string) ([]byte, error) {
	return f(filename)
}

From this point, the FileReaderFunc type implements the FileReader interface. The sole method it contains forwards the call to the instance of that type, which is the original method. This results in minimal changes when initializing the application:

New Main function

func main() {
	repository := NewConfigurationRepository(FileReaderFunc(ioutil.ReadFile))
	config, err := repository.GetConfiguration()
	//
	// do something with configuration
	//
}

And, it does not carry any change to unit test:

Test with FileReader

func TestGetConfiguration(t *testing.T) {
	var readerFunc FileReaderFunc

	// we want to have error from reader
	readerFunc = func(filename string) ([]byte, error) {
		return nil, errors.New("error")
	}

	repository := NewConfigurationRepository(readerFunc)
	_, err := repository.GetConfiguration()
	if err == nil {
		t.Error("error is expected")
	}

	// we want to have concrete result from reader
	readerFunc = func(filename string) ([]byte, error) {
		return []byte("content"), nil
	}

	repository = NewConfigurationRepository(readerFunc)
	config, err := repository.GetConfiguration()
	if err != nil {
		t.Error("error is not expected")
	}
	//
	// do something with config
	//
}

I prefer the second variation, as someone who is more inclined toward using interfaces and structs rather than independent functions. However, both of these solutions are valid choices.

Bonus 1: Mocking HTTP server

When it comes to mocking an HTTP server, I believe it goes beyond unit testing. However, there may be situations where your code structure depends on HTTP requests, and this section provides some ideas for handling such scenarios. Let’s consider a UserAPIRepository that sends and retrieves data by interacting with an external API rather than a database. This struct may look something like this:

UserAPIRepository

type UserAPIRepository struct {
	host string
}

func NewUserAPIRepository(host string) *UserAPIRepository {
	return &UserAPIRepository{
		host: host,
	}
}

func (r *UserAPIRepository) FilterByLastname(ctx context.Context, lastname string) ([]User, error) {
	var users []User

	url := path.Join(r.host, "/users/", lastname)
	response, err := http.Get(url)
	//
	// do somethinf with users
	//
	return users, nil
}

Naturally, we could also handle this by mocking functions, but let’s explore this approach. To create a unit test for UserAPIRepository, we can use an instance of Server from the core Go HTTPtest package. This package offers a simple local server that runs on specific ports and can be easily customized for our test cases, allowing us to send requests to it:

Test UserAPIRepository

import (
  	//
  	// some imports
  	//
	"net/http/httptest"
)

func TestUserAPIRepository(t *testing.T) {
	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		if strings.HasPrefix(r.URL.Path, "/users/") {
			var content string
			//
			// do something
			//
			io.WriteString(w, content)
			return
		}
		http.NotFound(w, r)
	}))

	repository := NewUserAPIRepository(server.URL)
	users, err := repository.FilterByLastname(context.Background(), "some last name")
	//
	// do some checking for users and err
	//
}

At this point, I would like to mention that a much better approach for testing integration with an external API is to use contract testing.

Bonus 2: Mocking SQL Database

Again, like for HTTP requests, I’m not particularly eager to write unit tests for testing SQL queries. I always question whether I’m testing a repository or a mocking tool. Still, when I want to check some SQL query, it is probably wrapped in some struct, like here in UserDBRepository:

UserDBRepository

type UserDBRepository struct {
	connection *sql.DB
}

func NewUserDBRepository(connection *sql.DB) *UserDBRepository {
	return &UserDBRepository{
		connection: connection,
	}
}

func (r *UserDBRepository) FilterByLastname(ctx context.Context, lastname string) ([]User, error) {
	var users []User

	rows, err := r.connection.Query("SELECT * FROM users WHERE lastname = ?", lastname)
	//
	// do something with users
	//
	return users, nil
}

When I decide to write unit tests for this kind of repositories, I like to use the package Sqlmock. It is simple enough and has excellent documentation.

Test UserDBRepository with Sqlmock

import (
	//
  	// some imports
  	//
	"github.com/DATA-DOG/go-sqlmock"
)

func TestUserDBRepository(t *testing.T) {
	db, mock, err := sqlmock.New()
	if err != nil {
		t.Error("expected not to have error")
	}

	mock.
		ExpectQuery("SELECT * FROM users WHERE lastname = ?").
		WithArgs("some last name").
		WillReturnError(errors.New("error"))

	repository := NewUserDBRepository(db)
	users, err := repository.FilterByLastname(context.Background(), "some last name")
	//
	// do some checking for users and err
	//
}

When mocking actual SQL queries becomes too exhausting, another approach is to use a small SQLite file with test data. This file should have the same table structure as our regular SQL database. However, this is not an ideal solution because we might test our queries on different database engines, and it’s better to depend on an ORM to avoid double integration. In this case, I create a temporary file and copy data from the SQLite file into it before each test execution. It is slower, but this way, I can avoid corrupting my test data.

Test data with SQLite

import (
	//
  	// some imports
  	//
	_ "github.com/mattn/go-sqlite3"
)

func getSqliteDBWithTestData() (*sql.DB, error) {
	// read all from sqlite file
	data, err := ioutil.ReadFile("test_data.sqlite")
	if err != nil {
		return nil, err
	}

	// create temporary file
	tmpFile, err := ioutil.TempFile("", "db*.sqlite")
	if err != nil {
		return nil, err
	}

	// store test data into temporary file
	_, err = tmpFile.Write(data)
	if err != nil {
		return nil, err
	}

	err = tmpFile.Close()
	if err != nil {
		return nil, err
	}

	// make connection to temporary file
	db, err := sql.Open("sqlite3", tmpFile.Name())
	if err != nil {
		return nil, err
	}

	return db, nil
}

Finally, the unit test looks much more straightforward now:

Test UserDBRepository with test data

func TestUserDBRepository(t *testing.T) {
	db, err := getSqliteDBWithTestData()
	if err != nil {
		t.Error("expected not to have error")
	}

	repository := NewUserDBRepository(db)
	users, err := repository.FilterByLastname(context.Background(), "some last name")
	//
	// do some checking for users and err
	//
}

Conclusion

Writing unit tests in Go can be more challenging compared to other languages, at least in my experience. It involves preparing the code to support the testing strategy. Surprisingly, I find this part enjoyable because it has helped me refine my architectural approach to coding more than any other language. It’s never boring, and there’s a constant sense of satisfaction, even after writing thousands of unit tests.

Useful Resources

• Martin Fowler
• Go Dev

• tutorial
• golang
• testing