Organizing Services with ZIO and ZLayers
In this article, we’ll take a look at ZLayer
s, an abstraction naturally arising from the core design of ZIO, which can greatly assist in making large code bases more understandable, composable and searchable for the human beings charged with their care.
This article is for the comfortable Scala programmer. Some familiarity with ZIO basics will help, but I’ll take care to outline the necessary concepts here so that the article can be as standalone as possible.
If you want to code with me in the article (or the YouTube video), you’ll have to add these lines to your build.sbt
file:
libraryDependencies ++= "dev.zio" %% "zio" % "1.0.4-2" // latest version at the moment of this writing
1. Background
The ZIO library is centered around the ZIO
type. Instances of ZIO are called “effects”, which describe anything that a program normally does: printing, computing, opening connections, reading, writing etc. However it’s worth pointing out that — much like other IO monads — constructing such an “effect” does not actually produce it at that moment. Instead, a ZIO instance is a data structure describing an effect.
The ZIO type describes an effect which is caused by an input, and can produce either an error or a desired value. As such, it takes 3 type arguments:
- an input type
R
, also known as environment - an error type
E
, which can be anything (not necessarily a Throwable) - a value type
A
and we thus have ZIO[-R, +E, +A]
. Conceptually, a ZIO instance is equivalent to a function R => Either[E,A]
, and there are natural conversion APIs between ZIO and the standard library structures.
This design allows instances of ZIO to be composed like functions, with various APIs, guarantees and conditions.
Some examples:
import zio.ZIO
// data structures to wrap a value or an error
// the input type is "any", since they don't require any input
val success = ZIO.succeed(42)
val fail = ZIO.fail("Something went wrong") // notice the error can be of any type
// reading and writing to the console are effects
// the input type is a Console instance, which ZIO provides with the import
import zio.console._
val greetingZio =
for {
_ <- putStrLn("Hi! What is your name?")
name <- getStrLn
_ <- putStrLn(s"Hello, $name, welcome to Rock the JVM!")
} yield ()
These ZIO instances don’t actually do anything; they only describe what will be computed or “done”. If we want the greetingZio
effect to actually run, we need to put it in a main app:
object ZioPlayground extends zio.App {
def run(args: List[String]) =
greetingZio.exitCode
}
Hello! What is your name?
> Daniel
Hello, Daniel, welcome to Rock the JVM!
Process finished with exit code 0
2. Services as Effects
In a real application, we often need to create heavyweight data structures which are important for various operations. The list is longer than we like to admit, but some critical operations usually include
- interacting with a database or storage layer
- doing business logic
- serving a front-facing API, perhaps through HTTP
- communicating with other services
Now, if we think about it, most of these data structures are created through some sort of effect: for example, creating a connection pool, reading from some configuration file, opening network ports, etc.
We can therefore conveniently think of these services as a particular kind of effect. ZIO matches this pattern perfectly:
- a service may have dependencies, therefore “inputs” or “environment”
- a service may fail with an error
- a service, once created, may serve as dependency or input to other services
This style of thinking about a service is the core idea behind a ZLayer
.
For the rest of this article, we’ll write a skeleton for an email newsletter service that automatically gives a user a welcome email, once subscribed. The implementations are console-based, but they can be easily replaced by a real database or a real email service. The goal of this example is to show you how to plug together independent components of your application.
3. The ZLayer
Pattern
Let’s assume we’re working with user instances of the form
case class User(name: String, email: String)
Let’s define a small service which, given a user, will send them a particular message to their email address. A simple API would look like this:
object UserEmailer { // service
trait Service {
def notify(user: User, message: String): Task[Unit]
}
}
A Task
is an alias for ZIO[Any, Throwable, A]
: produces a value (of type Unit
in this case), takes no inputs and can throw an exception.
An implementation of this service would send an email to this user, but for this example we’ll use a console printer:
val aServiceImpl = new Service {
override def notify(user: User, message: String): Task[Unit] =
Task {
println(s"Sending '$message' to ${user.email}")
}
}
The interesting thing is that, in order to make this service available to other parts of the application, we can wrap it inside an effectful creation of this service. This is where ZLayer
comes into play:
val live: ZLayer[Any, Nothing, Has[UserEmailer.Service]] = ZLayer.succeed(
// that same service we wrote above
new Service {
override def notify(user: User, message: String): Task[Unit] =
Task {
println(s"Sending '$message' to ${user.email}")
}
}
)
Much like ZIO
, a ZLayer
has 3 type arguments:
- an input type
RIn
, aka “dependency” type - an error type
E
, for the error that might arise during creation of the service - an output type
ROut
Note the output type in this case: we have a Has[UserEmailer.Service]
, not a plain UserEmailer.Service
. We’ll come back to this and show how this works and why it’s needed.
This live
instance sits inside the UserEmailer
object, as the live implementation of its inner Service
trait. Still inside the same object, it’s common to expose a higher-level API:
def notify(user: User, message: String): ZIO[Has[UserEmailer.Service], Throwable, Unit] =
ZIO.accessM(_.get.notify(user, message))
This may be hard to understand if you’re seeing ZIO
s for the first time. The notify
method is an effect, so it’s a ZIO
instance. The input type is a Has[UserEmailer.Service]
, which means that whoever calls this notify
method needs to have obtained a UserEmailer.Service
. If we do, then we can access that instance as the input of that ZIO instance, via accessM
, and then use that service’s API directly.
Here’s how we can directly use this in a main app:
object ZLayerPlayground extends zio.App {
override def run(args: List[String]): ZIO[zio.ZEnv, Nothing, ExitCode] =
UserEmailer
.notify(User("Daniel", "daniel@rockthejvm.com"), "Welcome to Rock the JVM!") // the specification of the action
.provideLayer(UserEmailer.live) // plugging in a real layer/implementation to run on
.exitCode // trigger the effect
}
So far, we have our first layer of our email newsletter service:
import zio.{ZIO, Has, Task, ZLayer}
// type alias to use for other layers
type UserEmailerEnv = Has[UserEmailer.Service]
object UserEmailer {
// service definition
trait Service {
def notify(u: User, msg: String): Task[Unit]
}
// layer; includes service implementation
val live: ZLayer[Any, Nothing, UserEmailerEnv] = ZLayer.succeed(new Service {
override def notify(u: User, msg: String): Task[Unit] =
Task {
println(s"[Email service] Sending $msg to ${u.email}")
}
})
// front-facing API, aka "accessor"
def notify(u: User, msg: String): ZIO[UserEmailerEnv, Throwable, Unit] = ZIO.accessM(_.get.notify(u, msg))
}
Another ZLayer in our email newsletter application can be a user email database. Following the same pattern, we arrive at a very similar structure:
// type alias
type UserDbEnv = Has[UserDb.Service]
object UserDb {
// service definition
trait Service {
def insert(user: User): Task[Unit]
}
// layer - service implementation
val live: ZLayer[Any, Nothing, UserDbEnv] = ZLayer.succeed {
new Service {
override def insert(user: User): Task[Unit] = Task {
// can replace this with an actual DB SQL string
println(s"[Database] insert into public.user values ('${user.name}')")
}
}
}
// accessor
def insert(u: User): ZIO[UserDbEnv, Throwable, Unit] = ZIO.accessM(_.get.insert(u))
}
4. Composing ZLayer
s
The two ZLayer
s we’ve just defined are so far independent, but we can compose them. Because the ZLayer
type is analogous to a function RIn => Either[E, ROut]
, it makes sense to be able to compose ZLayer
instances like functions.
4.1. Horizontal Composition
One way of combining ZLayers
is the so-called “horizontal” composition. If we have
- a
ZLayer[RIn1, E1, ROut1]
- another
ZLayer[RIn2, E2, ROut2]
we can obtain a “bigger” ZLayer
which can take as input RIn1 with RIn2
, and produce as output ROut1 with ROut2
. If we suggested earlier that RIn
is a “dependency”, then this new ZLayer
combines (sums) the dependencies of both ZLayer
s, and produces a “bigger” output, which can serve as dependency for a later ZLayer
.
For our use-case, it makes sense to combine UserDb
and UserEmailer
horizontally, because they have no dependencies and can produce a powerful layer which combines UserDbEnv with UserEmailerEnv
. In other words, there is such a thing as
val userBackendLayer: ZLayer[Any, Nothing, UserDbEnv with UserEmailerEnv] =
UserDb.live ++ UserEmailer.live
Remember what we wrote earlier when we used the email notification service directly?
override def run(args: List[String]): ZIO[zio.ZEnv, Nothing, ExitCode] =
UserEmailer
.notify(User("Daniel", "daniel@rockthejvm.com"), "Welcome to Rock the JVM!")
.provideLayer(UserEmailer.live) // <--- this is where we plug a ZLayer containing a real service implementation
.exitCode
We can replace UserEmailer.live
with this userBackendLayer
and it will still work. The nice thing is that this userBackendLayer
can also be directly used when we say
UserDb.insert(User("Daniel", "daniel@rockthejvm.com"))
.provideLayer(userBackendLayer)
.exitCode
so we can directly use this same “bigger” ZLayer
in both cases because it contains live implementations of both services.
4.2. Vertical Composition
Another way of composing ZLayer
s is by the so-called “vertical” composition, which is more akin to regular function composition: the output of one ZLayer
is the input of another ZLayer
, and the result becomes a new ZLayer
with the input from the first and the output from the second.
For our use-case, another ZLayer
might be more appropriate.
When a user signs up to our newsletter, we want to store their email in the database and send them the welcome email. In other words, we want to be able to invoke the two services from a third service, which will have a single, front-facing subscribe
API. We’ll start with the same pattern as before, but this time, we’ll implement the Service
as a class:
// type alias
type UserSubscriptionEnv = Has[UserSubscription.Service]
object UserSubscription {
// service definition as a class
class Service(notifier: UserEmailer.Service, userModel: UserDb.Service) {
def subscribe(u: User): Task[User] = {
for {
_ <- userModel.insert(u)
_ <- notifier.notify(u, s"Welcome, ${u.name}! Here are some ZIO articles for you here at Rock the JVM.")
} yield u
}
}
}
The difference here is that the inner Service
type doesn’t need any abstract methods since it only uses the other two services. Concrete instances of UserEmailer.Service
and UserDb.Service
will in turn influence the instances of UserSubscription.Service
via — you guessed it — dependency injection:
val live: ZLayer[UserEmailerEnv with UserDbEnv, Nothing, UserSubscriptionEnv] =
ZLayer.fromServices[UserEmailer.Service, UserDb.Service, UserSubscription.Service]( emailer, db =>
new Service(emailer, db)
)
This is a bit opaque and hard to read: where do the real instances of UserEmailer.Service
and UserDb.Service
come from?
If you remember the horizontal-composed ZLayer
:
val userBackendLayer: ZLayer[Any, Nothing, UserDbEnv with UserEmailerEnv] =
UserDb.live ++ UserEmailer.live
then we can use the output of userBackendLayer
as input of UserSubscription.live
. Here goes:
val userSubscriptionLayer: ZLayer[Any, Throwable, UserSubscriptionEnv] =
userBackendLayer >>> UserSubscription.live
We therefore obtain a single ZLayer
which contains the implementation of a UserSubscription.Service
, and the creation/passing of the UserEmailer.Service
and UserDb.Service
happens because of the construction of userBackendLayer
(which contains implementations for both) and the >>>
operator, which then calls the callback from ZLayer.fromService
. You don’t need to care about that, but that’s just if you’re curious (I for one was when I read on ZIO).
5. Plugging Everything Together
The final program to subscribe the first fan of Rock the JVM (me) to this fictitious email newsletter looks like this:
import zio.{ExitCode, Has, Task, ZIO, ZLayer}
case class User(name: String, email: String)
object UserEmailer {
// type alias to use for other layers
type UserEmailerEnv = Has[UserEmailer.Service]
// service definition
trait Service {
def notify(u: User, msg: String): Task[Unit]
}
// layer; includes service implementation
val live: ZLayer[Any, Nothing, UserEmailerEnv] = ZLayer.succeed(new Service {
override def notify(u: User, msg: String): Task[Unit] =
Task {
println(s"[Email service] Sending $msg to ${u.email}")
}
})
// front-facing API, aka "accessor"
def notify(u: User, msg: String): ZIO[UserEmailerEnv, Throwable, Unit] = ZIO.accessM(_.get.notify(u, msg))
}
object UserDb {
// type alias, to use for other layers
type UserDbEnv = Has[UserDb.Service]
// service definition
trait Service {
def insert(user: User): Task[Unit]
}
// layer - service implementation
val live: ZLayer[Any, Nothing, UserDbEnv] = ZLayer.succeed {
new Service {
override def insert(user: User): Task[Unit] = Task {
// can replace this with an actual DB SQL string
println(s"[Database] insert into public.user values ('${user.name}')")
}
}
}
// accessor
def insert(u: User): ZIO[UserDbEnv, Throwable, Unit] = ZIO.accessM(_.get.insert(u))
}
object UserSubscription {
import UserEmailer._
import UserDb._
// type alias
type UserSubscriptionEnv = Has[UserSubscription.Service]
// service definition
class Service(notifier: UserEmailer.Service, userModel: UserDb.Service) {
def subscribe(u: User): Task[User] = {
for {
_ <- userModel.insert(u)
_ <- notifier.notify(u, s"Welcome, ${u.name}! Here are some ZIO articles for you here at Rock the JVM.")
} yield u
}
}
// layer with service implementation via dependency injection
val live: ZLayer[UserEmailerEnv with UserDbEnv, Nothing, UserSubscriptionEnv] =
ZLayer.fromServices[UserEmailer.Service, UserDb.Service, UserSubscription.Service] { (emailer, db) =>
new Service(emailer, db)
}
// accessor
def subscribe(u: User): ZIO[UserSubscriptionEnv, Throwable, User] = ZIO.accessM(_.get.subscribe(u))
}
object ZLayersPlayground extends zio.App {
override def run(args: List[String]): ZIO[zio.ZEnv, Nothing, ExitCode] = {
val userRegistrationLayer = (UserDb.live ++ UserEmailer.live) >>> UserSubscription.live
UserSubscription.subscribe(User("daniel", "daniel@rockthejvm.com"))
.provideLayer(userRegistrationLayer)
.catchAll(t => ZIO.succeed(t.printStackTrace()).map(_ => ExitCode.failure))
.map { u =>
println(s"Registered user: $u")
ExitCode.success
}
}
}
6. So What’s With That Has
Thing?
We see that whenever we combine ZLayers
horizontally, we obtain inputs and outputs of the form Has[Service1] with Has[Service2]
. Why the Has[_]
? Why not just Service1 with Service2
à la cake-pattern?
If we had an instance of Service1 with Service2
, that single instance would have had both their APIs. On the other hand, Has[_]
is cleverly built to hold each instance independently while still maintaining the formal type definition. Strictly for our use case, an instance of Has[Service1] with Has[Service2]
has one instance of Service1
and one instance of Service2
, which we can surface and use independently, instead of a composite Service1 with Service2
instance.
7. Conclusion
We went through an overview of ZIO and we covered the essence of ZLayer
, enough to understand what it does and how it can help us build independent services, which we can plug together to create complex applications.
More on ZIO soon.