Idempotent mutable APIs and how they can help us

What are immutable and mutable APIs?

If we have an API, we generally can differentiate between immutable and mutable operations.

An immutable operation allows the caller to call the API without changing the state of the underlying data, while calling a mutable operation will change the data.

By right, all GET requests are immutable operations: We can call GET /user/{id} without changing the state data.

On the contrary, e.g. a POST request is mutable. If we use POST /users to create a new user, the state of the data will change as the new user is added to the dataset.

The problem with mutable APis

Let's assume we have a mobile app that can make money transfers from user to user via an endpoint POST /transfers.

By right, if all goes well, we make a request and the transfer is done and everybody is happy.

But what if my user has a spotty internet connection, and the request to the backend fails with a connection error?

Did my transfer go through or not? The server could have received and processed my request. Or it could have never received my request.

We can't check on the mobile app side, because we haven't gotten any response from the backend that would give us any id of the transfer we just did, so we could check.

Can my mobile app just retry the request? This may or may not work, because in case the request has actually gone through, my transfer will be duplicate! That is not acceptable.

What is Idempotency and how can it help us?

According to Wikipedia,

Idempotency is the property of certain operations in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application.

What does this mean for our APIs?

We can design our POST /transfers endpoint in a way that it will take an additional parameter like Txn_Request_UUID that is generated by the mobile app.

The backend will accept the request, perform the transaction and store the Txn_Request_UUID.

With this setup, our mobile app could safely retry a failed request, re-using the same Txn_Request_UUID. The backend will check the ID and only perform the transaction if the ID does not exist yet. If the ID already exists, it will return a success response.

This pattern goes beyond transactions, it could be applied for basically all mutable endpoints like e.g. POST /users, allowing all mutable endpoints to be safely retry-able.

Summary

There are some considerations though: Our endpoint's data manipulation must be fully transactional, otherwise we may run into problems when retrying, as parts of our initial request may be already processed, while others failed..

However, generally, this pattern can be very powerful, and while it is not a silver bullet, I am trying to implement it where it seems necessary and reasonable.