Testcontainers: what is it and how does it create better integration tests?
In daily software development, tests are an essential part of the process. By ensuring code coverage, we can mitigate issues, ultimately improving the quality of our code. In this scenario, Testcontainers presents itself as a valuable framework for integration testing, helping developers manage dependencies and create more reliable, reproducible test environments.
In this article, we’ll explore how testcontainers can be applied to real-world scenarios to improve the effectiveness of integration tests.
Applying tests comes with costs that should be evaluated. To illustrate this, Cohn (2009) defined a pyramid demonstrating how developers should approach writing tests.
According to Cohn (2009), tests are divided into three different categories, which are described as follows:
- Unit tests: Unit tests are the fastest and most isolated type of testing. This type of test focuses on a single portion of the code, typically testing just a specific function of interest to the developer. If one function calls another, developers usually create a mock of the called function to ensure the expected outcome from it.
- Service Tests or Integration tests: These tests typically assess a portion of the system as a whole. Unlike unit tests, integration tests do not mock the functions called within one another; instead, they evaluate the system in a continuous flow. Developing this kind of test is generally more resource-intensive than unit tests, taking more time to execute and sometimes requiring external dependencies, such as retrieving or storing data in a database.
- UI Tests or End-to-end (E2E) tests: These tests are the most expensive and can take a significant amount of time to execute. Developers rarely write this type of test because it can be very time-consuming, and they often lack the time needed to complete them.
On the other hand, developers are adopting more complex architectures that complicate the application of integration tests and end-to-end (E2E) tests. Today, many companies create software leveraging the microservices concept. To implement these types of tests, developers often need to create a separate environment to ensure that all dependencies are installed and accessible. However, this approach can sometimes be problematic, as multiple developers accessing the same data concurrently can lead to flaky tests due to dirty or unexpected data.
In light of this scenario, testcontainers can help address these challenges. But how can testcontainers contribute to creating better integration tests? This will be discussed in the following sections.
What is testcontainers?
According to the TestContainers portal, Testcontainers is an open-source library that provides containers to help isolate the data we need. The beauty of this technology is that developers can use it across various programming languages, including Golang, Java, Python, Rust, Elixir, Node.js, .NET, and more. Moreover, it greatly facilitates the implementation of integration and end-to-end (E2E) tests, as developers can define the dependencies they need, which will then be spun up automatically in an environment with Docker.
Testcontainers: How can developers apply it daily?
Consider a scenario where testing the user login journey within a system is necessary. In this journey, a new user should be created with a username and password, followed by field validation, and then stored in a PostgreSQL database. Additionally, the login endpoint needs to be verified to ensure it can fetch the stored user from the database and check if the password hash is correct, ultimately responding with whether access to the system is granted.
In this context, the system has a dependency on the PostgreSQL database, making it essential to conduct an integration test to evaluate the entire journey. If a specific environment with a dedicated database is not available, how can testing be performed effectively? This scenario illustrates how testcontainers can be utilized to address this challenge.
–> You may also like: Test-driven development in the real world: when to use it (and when not to)?
Testcontainers in action: creating a project structure to apply the integration test
The following example will be implemented using Golang; however, it is important to note that various programming languages can be utilized. A complete list of supported languages can be found directly on the Testcontainers portal, although some have already been mentioned in this article.
For this sample application, consider the following user table structure:
create table UserTable (
email varchar(100) not null,
password varchar(100) not null,
constraint pky_user primary key(email)
);
This simple table structure will be created in the database. To facilitate the creation of this table a migration package will be used. In this instance, the library https://github.com/golang-migrate/migrate will be employed, as it provides an easy way to set up the database infrastructure. The intention is to create a new container and apply the migration to establish an identical database structure as would be implemented in a real-world scenario.
The code structure will be kept simple to avoid complicating the understanding of the implementation. The following image illustrates the code structure.
Inside the use_case package are files that emulate the actions of the system. Notably, there is an insert_user.go file responsible for verifying whether a user exists; if the user does not exist, the system will store the new user in the PostgreSQL database. If the user already exists, the system will return an error. The following Golang code block describes the InsertUC file.
package use_case
import (
“context”
“errors”
“fmt”
“golang.org/x/crypto/bcrypt”
)
type InsertUc struct {
userRepo UserRepository
}
func NewInsertUC(repository UserRepository) *InsertUc {
return &InsertUc{
userRepo: repository,
}
}
func (i *InsertUc) InsertNewUser(ctx context.Context, user User) error {
if user.Email == “” || user.Password == “” {
return errors.New(“all field is required to save the user”)
}
uFound, err := i.userRepo.GetUserByEmail(ctx, user.Email)
if err != nil {
return err
}
if uFound != nil {
return fmt.Errorf(“there is an email %s stored in the database”, user.Email)
}
pwd, err := cryptUserPassword(user.Password)
user.Password = pwd
err = i.userRepo.CreateUser(ctx, user)
if err != nil {
return err
}
return nil
}
func cryptUserPassword(password string) (string, error) {
fromPassword, err := bcrypt.GenerateFromPassword([]byte(password), bcrypt.MinCost)
if err != nil {
return “”, err
}
return string(fromPassword), nil
}
Inside the login.go file, there is a login function that verifies whether the provided username and password are valid. The following code block presents the LoginUC structure.
package use_case
import (
“context”
“errors”
“fmt”
“golang.org/x/crypto/bcrypt”
)
type LoginUc struct {
userRepository UserRepository
}
func NewLoginUc(userRepository UserRepository) *LoginUc {
return &LoginUc{userRepository: userRepository}
}
func (l *LoginUc) Login(ctx context.Context, email, password string) (*User, error) {
if email == “” || password == “” {
return nil, errors.New(“email or password is empty”)
}
user, err := l.userRepository.GetUserByEmail(ctx, email)
if err != nil {
return nil, err
}
err = bcrypt.CompareHashAndPassword([]byte(user.Password), []byte(password))
if err != nil {
return nil, fmt.Errorf(“password for email %s isn’t valid”, email)
}
return user, nil
}
In the repository package, there is a file named user_repository.go. This file manages the access from the system to the PostgreSQL database, containing SQL instructions for inserting and retrieving users. The next code block presents the code for the mentioned file.
package repository
import (
“context”
“errors”
“github.com/jackc/pgx/v5”
“testcontainer/use_case”
)
type UserRepository struct {
conn *pgx.Conn
}
func NewUserRepository(conn *pgx.Conn) *UserRepository {
return &UserRepository{conn: conn}
}
func (ur *UserRepository) CreateUser(ctx context.Context, user use_case.User) error {
_, err := ur.conn.Exec(ctx, “insert into usertable(email, password) values ($1, $2)”, user.Email, user.Password)
if err != nil {
return err
}
return nil
}
func (ur *UserRepository) GetUserByEmail(ctx context.Context, email string) (*use_case.User, error) {
u := use_case.User{}
err := ur.conn.QueryRow(ctx, “select * from usertable where upper(email) = upper($1)”, email).Scan(&u.Email, &u.Password)
if err != nil && !errors.Is(err, pgx.ErrNoRows) {
return nil, err
}
if u.Email == “” {
return nil, nil
}
return &u, nil
}
To test the login journey, an integration test will be created to insert a new user and call the login function. In the next section, testcontainers will be used to spin up a PostgreSQL database using a Docker container.
Creating a new testcontainer and testing the Login UC journey
From this point, an integration test will be created to cover the login journey. To test the login functionality, a user must be stored in the database. The approach will involve creating a container that runs PostgreSQL to store the user, enabling retrieval and password comparison afterward. The following code block includes the code to spin up the PostgreSQL container and create the database structure using the previously mentioned migration package.
func spinUpDependencies(t *testing.T, ctx context.Context) (testcontainers.Container, error) {
req := testcontainers.ContainerRequest{
Image: “postgres:17”,
ExposedPorts: []string{“5432/tcp”},
Env: map[string]string{
“POSTGRES_DB”: “userdb”,
“POSTGRES_USER”: “pgsql”,
“POSTGRES_PASSWORD”: “root”,
},
WaitingFor: wait.ForAll(wait.ForListeningPort(“5432/tcp”)).
WithStartupTimeoutDefault(60 * time.Second),
}
container, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: req,
Started: true,
})
if err != nil {
t.Fatalf(“could not start container: %v“, err)
}
return container, nil
}
In the previous code block, it is possible to observe that testcontainers split the container initialization into parts. The first part, ContainerRequest, provides the configuration for the container. It specifies the image to be used, the ports to be exposed, the environmental variables expected by the container, and a notable parameter called WaitingFor. This special parameter monitors the logs or specific container events to determine when to proceed with code execution. In this case, the code is released as soon as the port begins listening; if it does not, a timeout will occur after 60 seconds.
After configuring the settings, containers.GenericContainer is called to start the container based on the provided configuration. This is where any potential errors can be captured, or a reference to the container can be obtained.
Creating the user table to store user data in Testcontainers
After creating the PostgreSQL container, the next step is to migrate the SQL instructions that need to be executed. For the context of this article, the focus will be on creating a single table named UserTable. The following code block illustrates the migration function.
func migrateData(username, password, host, databaseName string, port int) error {
databaseUrl := fmt.Sprintf(“postgres://%s:%s@%s:%d/%s?sslmode=disable”, username, password, host, port, databaseName)
pgxConfig, err := pgx.ParseConfig(databaseUrl)
if err != nil {
log.Fatalf(“Unable to parse PostgreSQL config: %v\n“, err)
}
stdlib.RegisterConnConfig(pgxConfig)
db := stdlib.OpenDB(*pgxConfig)
defer db.Close()
m, err := migrate.New(“file://../migrations”, databaseUrl)
if err != nil {
return err
}
err = m.Up()
if err != nil && !errors.Is(err, migrate.ErrNoChange) {
return err
}
return nil
}
The migration function responsible for creating the table is presented below. The simple SQL code that will be executed can be seen in the following code block.
create table usertable(
email varchar(100),
password varchar(100),
constraint pky_usertable primary key(email)
);
Integration testing with Testcontainers: validating user creation and login flow
Finally, the integration test execution will be responsible for testing the respective journeys. In the context of this article, a simple scenario has been chosen to demonstrate the test container itself. However, it becomes particularly powerful when developers have numerous internal dependencies that must be configured to test a specific journey within a system. The following code block presents the function that tests the expected journey in a simplified manner.
func Test_IntegrationLoginTest(t *testing.T) {
ctx := context.Background()
container, err := spinUpDependencies(t, ctx)
if err != nil {
t.Fatalf(“could not spin up dependencies: %v“, err)
}
defer container.Terminate(ctx)
host, err := container.Host(ctx)
if err != nil {
t.Fatalf(“could not get host container: %v“, err)
}
port, err := container.MappedPort(ctx, “5432”)
if err != nil {
t.Fatalf(“could not get port container: %v“, err)
}
pgClient := repository.NewPGClient(ctx, “pgsql”, “root”, host, “userdb”, port.Int())
if pgClient == nil {
t.Fatalf(“couldnt create client for postgres database”)
}
err = migrateData(“pgsql”, “root”, host, “userdb”, port.Int())
if err != nil {
t.Fatalf(“could not migrate data: %v“, err)
}
userRepo := repository.NewUserRepository(pgClient.GetConn())
insertUC := use_case.NewInsertUC(userRepo)
err = insertUC.InsertNewUser(ctx, use_case.User{
Email: “ecore@ecore.com”,
Password: “123123”,
})
if err != nil {
t.Fatalf(“could not insert user: %v“, err)
}
loginUc := use_case.NewLoginUc(userRepo)
login, err := loginUc.Login(ctx, “ecore@ecore.com”, “123123”)
if err != nil {
t.Fatalf(“could not login: %v“, err)
}
if login.Email != “ecore@ecore.com” {
t.Fatalf(“invalid login”)
}
}
The previous code block demonstrates the complete function for spinning up the dependencies, migrating the necessary SQL components, and executing the expected function. The goal is to complete this function without encountering any fatal errors or issues.
Testcontainers: a powerful solution for high-quality integration testing
Despite the simple scenario presented and the use of just one container, testcontainers have proven to be a powerful tool for ensuring better system quality through integration testing. Depending on the situation, they can also be beneficial for end-to-end (E2E) tests, as testcontainers can utilize Dockerfiles to create various containers.
In a landscape where Continuous Integration is a best practice and Docker has become a market standard, testcontainers can be adopted to enhance both the speed and quality of software development.
To provide readers with access to the complete code, the entire repository is available for download and analysis.
Reference: The entire repository of this paper can be found here: https://github.com/zuchi/testcontainer
Jederson Zuchi
Senior Software Engineer
LET'S CONNECT
Ready to unlock your team's potential?
e-Core
We combine global expertise with emerging technologies to help companies like yours create innovative digital products, modernize technology platforms, and improve efficiency in digital operations.
You may also be interested in:
