Dependency injection – superb flexibility within monoliths

Dependency injection is a software design where a component does not initialise its dependencies, but these dependencies are passed into it instead. If done right, it greatly improves flexibilty and testability. This article tells how to do it right and reap the benefits.

First, what is it and how does it differ from a more traditional structure?

Because speaking of objects A, B and C feels too abstract, let’s think of an example app that allows the user to scan the barcode of a snack, showing instantly its calorie content (that may be hard to find on the packaging). It accesses the camera, looks for a barcode, sends it to a server, receives a response about the snack’s calories, caches it to preserve server bandwidth and shows the information to the user. And let us pretend it’s actually a complex task that cannot be done by a single person in a few days.

The natural design would be to have the UI initialise the camera and subscribe to its pictures, calling a function refreshing the image shown by the UI and feeding it into a barcode scanning library. It would use the barcode scanner’s results to query a barcode manager component for the calorie value and show it. The barcode manager would initialise and a network conneciton object and use it to send a query to the server and then save and return the result.

With dependency injection, the components are not different, but they are grouped differently. There are interfaces for camera, barcode scanner, calorie information provider, persistent data storage and networking. The camera interface can be subscribed to to provide new picture events. The barcode scanner can be subscribed to to provide barcodes. The calorie information provider has a method taking bacodes and returning nutritional information. The persistent data storage allows storing some data and returning it in a following run. The networking interface returns some kind of connection object.

So the main function (or some other important function) first creates a camera object, then uses it to construct the barcode scanner. Then it creates a networking interface and uses it to construct the calorie information provider. Then it creates the UI, giving its constructor the instances of the camera, barcode scanner and calorie information provider.

Camera camera = getFirstCamera();
BarcodeScanner barcodeScanner(camera);
MainServerConnection network("gimmethecalorieinfo.notarealdomain.com");
PersistentDataStorage storage("barcodesAndCalories.csv");
CalorieInformationProvider infoProvider(network, storage);
UI ui(camera, barcodeScanner, infoProvider);
ui.run();

The code is written in C++, but nothing of it is C++ specific. It could be any object-oriented programming language after some changes to syntax. If you don’t understand this code, look here.

What is the benefit of this?

The most obvious one is development. It’s annoying to try it out and always initialise the camera and put a barcode in front of it. Having to clear the data saved on disk can also be annoying. We might need to run it in some special code analyser that butchers performance and is unusable with image processing. The UI itself may be inconvenient for testing other parts of the system. So we might have simple alternative implementations of the interfaces, a fake camera that outputs the same picture periodically or when requested, a fake barcode scanner that simply outputs a given value and a fake storage that just wraps around a hashtable with some dummy barcodes and calorie values. So changing the function that sets it up can make the system run with far more convenient requirements without touching the code that is developed.

If some non-essential library is difficult to install in a development mode, is annoyingly big or incompatible with something else, a fake could allow developing parts not related to this library without having to install it.

This also improves configurability. An alternate barcode scanner library can have the same interface and the setup function can read a configuration file and choose which one to create and provide to the UI. Networking can be disabled by replacing the networking component with a dummy one that always returns connection errors.

Automated testing also benefits from this. Most of the app’s internals can be tested as unit tests (or almost unit tests), using the fake camera, fake persistent storage and fake network to run without depending on other processes.

Another benefit of dependency injection is that initialisation is necessarily very explicit and thus transparent and surprises like realising too late that some object wasn’t initialised are rare.

Interfaces, implementations, fakes, mocks

With the term interface, I mean any entity that allows multiple implementations to be used in the same way. It can be an interface like in Java, an abstract class, a protocol like in Swift, a C structure with a bunch of function pointers (useful for interaction between components written in different programming languages), a concept like in C++20, etc. It can even be a utility class wrapping around a small object with abstract methods only, providing utility to the overly general abstract methods. If this interface needs to be created multiple times, then it has to be a factory that returns the interface that is needed multiple times (in that case, the factory can be the only function provided by a dynamically linked library). Its purpose is to allow injecting different implementations into the constructor, so it shouldn’t be overly specialised and shouldn’t contain dozens of functions that have to be implemented (you can use a facade for that).

Overridable functions come with a certain performance cost, so they shouldn’t be used in the innermost loops. This problem can be avoided by giving ranges to these functions and having them do the iteration as well.

An implementation is a class doing the work represented by the interface. It may be potentially a large system, composed of a lot of components of its own, requiring its own set of dependencies, or may even have its own dependency injection system. All of its heaviness (lots of symbols coming from dependencies, slow compilation) is hidden from the rest of the codebase by the interface.

A fake also implements the interface, but does it in a greatly simplified way, but without changing the logic of its usage. It has minimum dependencies, communicates with nothing and doesn’t contain much code. Its main purpose is to allow running the rest of the program without the component whose interface it implements.

A mock is similar to a fake, but its behaviour is meant to follow some testing plan. It typically returns queued replies and checks if the input arguments are as expected. This is very useful in unit tests. There are frameworks for creating mocks for interfaces, but they usually don’t implement any internal logic beyond queuing expected calls. Various hybrids between mocks and fakes can exist, for example as classes implementing the interface but with everything public to allow testing the logic performed by the tested component on them rather than verifying individual calls.

Examples of components and their fakes and mocks:

• Connection to a device
  • A mock returns one of a bunch of prepared responses to expected requests
  • A fake (less convenient) has an instance of some part of the device’s software and gets a response from it
• Device communication component (has functions that request values from the server)
  • A mock returns one of prepared responses to expected calls
  • A fake simulates some basic behaviour and returns possible results
• Cache for some data
  • A mock expects some data to be inserted and some data to be retrieved
  • A fake wraps around some kind of table with all values inserted so far and can retrieve them (doesn’t scale, but gives the same appearance)
• Database (SQL)
  • A mock expects some queries and holds prepared responses for them
  • A fake implementing an ability to parse SQL would be too much for a fake, but it might replace some more complex database by SQLite (possibly erased at startup)
    • This layer should better not use a fake, a fake should be on a layer with functions that do the queries and return results
• Graphical User Interface input – a GUI typically controls the entire program, so it’s unlikely to ever be a dependency
  • A mock will replay some prepared sequence of widget inputs from a supposed user and check if the wanted values and widgets are ordered to be displayed
  • A fake makes sense only if there were multiple users and it was useful to simulate the activity of other users
• Command Line Interface input – a CLI typically controls the entire program, so it won’t ever be a dependency
  • A mock will replay some prepared sequence of commands from a supposed user and record or check the responses
  • A fake makes sense only if there were multiple users

Don’t overdo it

Only larger components need to be replaceable. Most classes can be just normal classes. A well designed codebase has a large number of classes and making fakes or mocks for them is too much work for little benefit.

It makes sense only to replace entire layers of code or where alternate implementations are reasonably expectable.

What about singletons?

Singleton is a controversial design pattern that is capable of turning a codebase into some kind of pasta. I am calling it a design pattern because it’s better than a bunch of static functions haphazardly manipulating some global variables. Acceptable usages include various loggers – logging is used everywhere and it’s annoying to put references to it into everything. Recording some usage or performance statistics can also be considered some kind of logger. The log may be readable internally in the program to show some information to the user (however, the logger should not directly call a GUI function).

A singleton does not need to be injected into objects, but it can be written in a way that allows replacing it with fakes or alternative implementations (which may be useful if a logger needs to communicate with an external process or executes files from the Internet). While a typical singleton will have private static instance that is created when first accessed, it also may be an abstract class whose instance is created somewhere else (or a default one can be replaced by an alternative). This allows some privileged function to set up the singleton as needed, but it has to be done for the entire process.

Circular dependencies

Circular dependencies will sometimes happen. Class A uses class B and caches some information, then if class B changes the information class A may have cached, it needs to call a method of class A. This problem can be solved by adding a method to class B for registering callbacks that would be called when needed (changing data used by the cache), and have A use it to register its method (that clears cache). It may also be a small interface defined near B implemented by A.

What about microservices?

Microservices are often used in place of dependency injection. They are not bad by themselves and can be used together with dependency injection. However, overusing them breaks a system that can be monolithic into small pieces, making the startup extremely complicated, dependent on scripting that grows, becomes a complex system on its own with dependencies and dockers and ends up heavily limiting development flexibility and ease of use. With dependency injection, the whole setup variability can exist within a single executable in a single process.

Of course, if there is a need to split the system into multiple processes (e.g. it needs to run on multiple computers), then microservices are a totally good approach. It’s much better than launching ssh sessions to execute commands or similar approaches.

More detailed example: A video game

I am not going to go into details what game I have in mind, let’s just say that it has singleplayer and networked multiplayer mode, some graphics (not text based), some combat and doesn’t use a game engine that would prevent Dependency Injection (which many do).

It would probably break down to components like these:

• User input
• GUI
• User commands
• AI
• Communication
• Game state
• Networking
• Game mechanics
• Visualisation
• Resources
• Sound
• Renderer
• Settings
• Log

The game state is an internal representation of the situation in the game, a rough equivalent of the stuff on the table in a tabletop game. It is influenced mainly by game mechanics, but also by user commands and AI to start or queue their actions, communication so that the actions of other players could influence the game, by cutscenes or by a console. It should also contain information about the movement or planned actions, so that the situation sent by other users a short time ago can be predicted to the present state. Its boundary with game mechanics may be somewhat difficult to determine, because some interaction may be derived from the situation, for example if a character positioned in water is automatically slower, the decrease of speed is a game mechanic yet it can be determined by the game state (in this situation, it should contain the information that the character is in water and that he is slower, but game mechanics decides he’s slower and how much slower he is). Serialising it (or some of it) saves the game, deserialising it loads the game, which is a useful feature even if the game has a checkpoint system, for debugging purposes. It is not one big class, it contains many classes internally to represent characters, items, weapons, attacks and so on. Because it mostly stores relatively small volumes of data in RAM and doesn’t depend on much stuff, it doesn’t really need to be faked or mocked.

struct Attack : StateObject { // The StateObject parent class helps synchronisation
	float speed = 1.0;
	float damage = 25.0;
	// blablabla
};

struct Obstacle : StateObject {
	FloatVector coordinates = {0, 0, 0};
	FloatVector speed = {0, 0, 0};
	// ...
};

struct Combatant : Obstacle {
	float health = 1.0;
	float maxHealth = 100;
	std::vector<Attack*> attacks = {};
	std::vector<Attack*> attackingQueue = {};
	Attack* currentAttack = nullptr;
	// blablabla
};

struct Projectile : StateObject {
	Attack& attack = 0;
	// blablabla
};

struct GameState : StateObject {
	std::vector<Combatant*> combatants = {};
	std::vector<Projectile*> projectiles = {};
	// blablabla
};

To focus on the topic, these code stubs mostly neglect threading, serialisation and memory management. All of this would need some additional wrapper classes. They are in C++, but the idea would work in other languages with some syntactic changes. If you are too unfamiliar with C++ and don’t understand these declarations, look here.

The game mechanics is the component taking care of the rules of the game. It controls the interaction of actors appearing in the game. It contains classes wrapping around the classes from game state, but with functionality related to game mechanics, but these don’t need to be faked or mocked. It takes input from time ticks, and applies its changes to the game state. Faking this is useful for viewing the game world without actually playing the game, performing movement actions from the player’s input but nothing else. A different fake may used to record the decisions of AI when unit testing it.

class CombatantMechanics : public Mechanic {
	Combatant* state = nullptr;
	std::vector<Attack*> attacks = {};
	// blablabla
	
public:
	void tick(milliseconds time) override {
		state->position += state->movement * time;
		
		if (state->currentAttack == nullptr && state->attackingQueue.size() > 0) {
			state->currentAttack = state->attackingQueue.front();
			state->attackingQueue.pop_front();
		}
		// blablabla
	}
};

// ...

class GameMechanics : IGameMechanics {
	GameState& state;
	std::vector<Mechanic*> mechanics = {};
	// ...
	
	void updateMechanics() {
		// Somehow finds what objects were added and adds mechanics for them
		// Somehow finds what objects were removed and removes their mechanics
	}

public:
	GameMechanics(GameState& state)
		: state(state) {} // In a different language, it would be this.state = state
		
	void tick(milliseconds time) override {
		updateMechanics();
		for (auto mechanic : mechanics) {
			mechanic->tick(time);
		}
		// blablabla
	}
};

Communication works with the game state, updating it according to the actions of other users and sends them the changes made by the player. It uses the networking component to communicate. This component is absent in single player, where no remote actor changes the game state. If the program is configured as a host, his game state is propagated by this component to other players, but after accepting the changes done to the characters under their control. If it’s connected to a host or a server, then it applies all changes to the local game state. Turn-based games might do this differently, use no server or host but propagate the players’ inputs and having all clients compute all the mechanics independently, perfectly synchronised. If the rest of the game can run without this, then it doesn’t even need to be faked, otherwise it might be faked to provide singleplayer mode.

class ClientCommunication {
	GameState& gameState;
	Networking& networking;
	Side playerSide;
	
	void updateState(Message& message) {
		// Identifies what part of state it updates and replaces the value
	}
	
public:
	ClientCommunication(GameState& gameState, Networking& networking, Side playerSide)
		: gameState(gameState), networking(networking), playerSide(playerSide)
	{
		networking.subscribeToClientUpdates([&] (const Message& message) {
			updateState(message);
		});
	}
	
	void tick(milliseconds time) {
		// A smarter code would of course check if there is something new
		for (auto combatant : gameState.combatants) {
			if (combatant.side == playerSide && combatant.player()) {
				networking.sendUpdate(combatant.serialiseCommands());
			}
		}
	}
	
	// ...
};

The networking component sends the messages from communication to other users or server. It should not care about the meaning of the messages, but there is no clear distinction whether it serialises the messages or communication does. Faking this allows simulating the actions of another player without actually connecting two instances, or without connecting the two instances within the same process through a real network.

struct INetworking {
	virtual Subscription subscribeToClientUpdates(std::function<void(const Messsage&)> reaction) = 0;
	virtual void sendUpdate(const Message& update) = 0;
	
	// For some transactions that need confirmations
	virtual Subscription subscribeToRequests(std::function<Message(const Message&)> callback) = 0;
	virtual Message requestResponse(const Message& update) = 0;
	
	// ...
};

Visualisation takes care of the graphical representation of the game state. There are many libraries that do most of the work, but they are too universal to be directly used by the game state. There has to be a layer specialised for the game in question. If game state‘s say that a character is wearing a blue shirt, then visualisation decides that shirt mesh is attached to its skeleton, it’s using the shirt material with colour variant blue (the material should be abstract enough because this layer doesn’t know what kind of shaders does the renderer use). If game state‘s data say that character A is attacking and that attack has started 0.1 seconds ago and is going to take 0.3 seconds, then visualisation has to decide which animation is representing the attack at what time the animation is at that time. This component would contain lots of classes representing the visuals of various objects in the game, possibly with dependency injection. If building the scene is not fast enough, it may need to be able to hold multiple scenes and switch between them quickly. It is needed by game mechanics for collision detection and possibly determining what surfaces players are on. It’s also needed to determine what a player clicked on. Faking this is not particularly useful, because there is no better way to tell what’s going on in the game (debugging would benefit more from making walls transparent). A fake could provide a simplified collision detection by detecting collisions only with height 0 as ground and maybe simplifying actors’s shapes to spheres, which is quite simple to compute.

class Visualisation : public IVisualisation {
	Renderer& rendeder;
	GameState* gameState = nullptr;
	//... the information about the scene

public:
	Visualisation(Renderer& renderer, GameState* gameState = nullptr)
		: renderer(renderer), gameState(gameState) {}

	void updateScene(milliseconds time) {
		// Run while the GPU is rendering the scene
		// Alters the scene in Renderer accordingly to the situation in gameState
	}
	
	std::optional<FloatVector> checkCollision(Obstacle& moving, milliseconds time) override {
		// Check if the object would bump into something
	}
	
	Obstacle* objectAt(ScreenCoordinates coordinates) override {
		// Picks an object on screen, returns null if there's nothing or it's not interactible
	}
	
	// ...
};

Resources are needed by the visualisation part to decide about collisions and provide data needed by the visuals and sound. It might also handle data about levels, cutscenes, characters and so on. It’s not a trivial load from disk because the game needs to preload data to avoid short waits in rendering (stuttering) when they are suddenly needed and unload data to avoid filling up the RAM. A fake resources component would provide just a few basic resources (for example one for each type) and keep everything in memory, or avoid loading textures to quicken loading, but it might not be needed at all.

struct IResources {
	virtual Resource<Model> getModel(std::string name) = 0;
	virtual Resource<Texture> getTexture(std::string name) = 0;
	virtual Resource<Mesh> getMesh(std::string name) = 0;
	virtual Resource<Sound> getSound(std::string name) = 0;
	// ...
};

The renderer is typically part of a graphics library, providing the access to the GPU driver. These tend to be replaceable regardless of the actual design of the library, because the game may need to use Vulkan, DirectX or Metal for graphical acceleration. A fake renderer doing nothing is also useful for servers so that they don’t need a GPU (the visualisation is still needed for collision detection). Only visualisation is going to use it.

A component from sound makes the user hear sounds according to visualisation and music. Its fake would simply produce no sound and possibly give a fixed duration for sounds if it’s meant to provide that.

struct ISoundSystem {
	virtual void playSound(std::string name, FloatVector origin) = 0;
	virtual void playUiSound(std::string name) = 0;
	virtual std::function<void()> playCancellableSound(std::string name, FloatVector origin) = 0;
	// ...
};

The AI component controls bots or non-playable characters. It inputs its actions to game state so that game mechanics processes them, identically to user commands. It may need game mechanics to estimate the value of different targets. It takes input from the game state and possibly visualisation (to decide if the character can see some other characters and determine paths to targets). It would probably be a complex class, possibly with many subclasses and possibly dependency injection on its own. The game should be able to run without this (with all characters idle), so the only need to fake this would be to reduce CPU cost by having the AI simply attack if a valid target is nearby, maybe going towards it if there is a direct path.

class Ai : public IAi {
	GameState& gameState;
	// Some internals
	
public:
	Ai(GameState& gameState)
		: gameState(gameState) {}
		
	void ponder() override {
		// Do the AI stuff, queuing some movement and actions at the end
	}
};

User commands does the same as AI, but makes decisions according to the player rather than the game state. It takes the user’s commands from user input. Keybinding may be done on this level. It probably needs alternative implementations for joysticks, VR controllers etc. Its fake is the AI.

class UserCommand : public IAi {
	GameState& gameState;
	UserInput& userInput;
	Combatant* controlled = nullptr;
	Subscription mainAttack = {};

public:
	UserCommand(GameState& gameState, UserInput& userInput)
		: gameState(gameState), userInput(userInput) {
		mainAttack = userInput.subscribeToMainAttack([&] () {
			// Queue the main attack here
		});
	}
	void setCombatant(Combatant* assigned) { combatant = assigned; }

	void ponder() override {
		combatant->speed = userInput.getMovement();
	}
	// ... 
};

User input allows the GUI and user commands to know what buttons the user is pressing, how he’s moving the mouse or where he’s clicking. A joystick may or may not have a different user input class. This may need to be mocked in unit tests to go through prepared sequences of user inputs to test if the user commands component produces the correct actions.

struct IUserInput {
	virtual FloatVector getMovement() = 0;
	virtual Subscription subscribeMainAttack(std::function<void()> callback) = 0;
	virtual Subscription subscribeSecondaryAttack(std::function<void()> callback) = 0;
	// ...
};

GUI is a high level component that injects widgets into the visualisation and may interpret clicks on them or button presses when they are active. It can display information from the game state that might not be very visible in the graphical output but the player should be aware of (such as health of characters), using visualisation to determine where to position it and what to show, so that it would be matched with the scene. It can make various changes into game state, settings and can display some contents of the log, placing it high in the hierarchy. It decides if the user’s actions propagate to user commands. It may have interfaces and fakes on its own to simulate user clicks on widgets. A fake GUI would display nothing and propagate all actions to user commands, if there is a need for that.

Settings is a set of mostly data classes representing the configuration. It takes care of persisting them between runs and possibly notifying other classes about changes. It should not define specific variables used by individual components. If it doesn’t do anything else, it doesn’t need to be faked.

Log is (probably) a singleton class used by everything to collect information about what is going on. It may have severity levels, some messages should be directly viewable through GUI, others just in a file or in a console. Some statistical information (such as the framerate) should have a specific non-text buffer. Being a class to mostly hold data, and crucial to function while debugging it doesn’t really need to be faked.

Appendix: Look here if you can’t read C++

For those unfamiliar with C++, here are some features of C++ that differ from many other object-oriented languages:

• Declaring a variable of a class type constructs it, using the arguments in parentheses behind the declaration
• Variables are deepcopied by default, an ampersand after the type (`BarcodeScanner&) is needed to make the declared variable a reference
  • A pointer is sometimes needed as some kind of nullable and changeable reference (declared with an asterisk, as BarcodeScanner*), access is then through -> instead of .
• Constructors don’t have an identifying keyword, they look like functions with the same names as their classes, with no return value
• Methods are final by default, to make a function overridable, it has to be declared with the virtual keyword
  • Generic functions cannot be overridable
    • Abstract classes can be generic, though
• The usual extendable array type is std::vector
  • It’s the fastest container type, so it’s often used for any type of collection
    • Yes, despite having a O(n) complexity in many operations, it beats other structures if there aren’t hundreds of elements
• There is no distinction between abstract classes and interfaces
  • A class can have any number of parent classes (it’s therefore possible to inherit from two classes with the same parent)
• Public attributes are not considered bad in data-only classes
  • Using the struct keyword instead of the class keyword makes the contents public by default

Appendix 2: A C++ specific problem with interfaces

In C++, it’s not possible to return an abstract class, which makes abstract factories less convenient. A function returning an object constructs it at the location of the return address, so the exact type to be returned must be in the function’s signature. This can be avoided by dynamically allocating it (e.g. with new or something smarter) and returning a pointer, but it comes at a performance cost of worsened memory locality and the allocation itself that may not be neglectful (other languages typically dynamically allocate all objects). There are multiple solutions, neither of them works in all cases, so sometimes, dynamic allocation has to be used:

• Use the factory as a template argument (you may use an interface at some level to allow one code to use all instantiations)
• Use a fixed-size buffer to allocate it and use dynamic allocation only if it doesn’t fit (std::function does this)
• Don’t return it, have the factory create it and call a callback on it:

factory.sender([&] (Sender& sender)
	sender.send(stuff);
);