One of the more powerful features Scala has is the ability to generically abstract across things that take type parameters. This feature is known as Higher Kinded Types (HKT).
This feature allows us to write a library that works with a much wider array of classes, whereas without the feature you are condemned to bespoke and error ridden code duplication for each class that may want the functionality.
Type constructors
Essentially what HKT gives us is the ability to generalize across type constructors – where a type constructor is anything that has a type parameter. For instance List[_]* is not a type, the underscore is a hole into which another type may be plugged, constructing a complete type. List[String] and List[Int] being examples of complete (or distinct) types.
Kinds
Now that we have a type constructor we can think of several different kinds of them, classified by how many type parameters they take. The simplest – like List[_] – that take a single param have the kind:
(* -> *)
This says: given one type, produce another. For instance, given String produce the type List[String].
Something that takes two parameters, say Map[_, _], or Function1[_, _] has the kind:
(* -> * -> *)
This says: given one type, then another, produce the final type. For instance given the key type Int and the value type String produce the type Map[Int, String].
Furthermore, you can have kinds that are themselves parameterized by higher kinded types. So, something could not only take a type, but take something that itself takes type parameters. An example would be the covariant functor: Functor[F[_]], it has the kind:
((* -> *) -> *)
This says: given a simple higher kinded type, produce the final type. For instance given a type constructor like List produce the final type Functor[List].
Utility
Say we have some standard pattern for our data-structures where we want to be able to consistently apply an operation of the same shape. Functors are a nice example, the covariant functor allows us to take a box holding things of type A, and a function of A => B and get back a box holding things of type B.
In Java, there is no way to specify that these things share a common interface, or that we simply want transformable boxes. We need to either make this static eg. Guava’s Lists and Iterables, or bespoke on the interface, eg: fugue’s Option or atlassian-util-concurrent’s Promise. There is simply no way to unify these methods on either some super interface or to specify that you have – or require – a “mappable/transformable” box.
With HKT I can represent the covariant functor described above as:
[cc lang=’scala’ ]
trait Functor[F[_]] {
def map[A, B](fa: F[A])(f: A => B): F[B]
}
// implement for java’s List
// note that the presence of mutation in the Java collections
// breaks the Functor laws
import java.util.{ List => JList }
implicit object JavaListFunctor extends Functor[JList] {
import collection.JavaConverters._
def map[A, B](fa: JList[A])(f: A => B): JList[B] =
(for (a B): Box2[B] =
Box2(f(b.a1), f(b.a2))
}
// and use it**
def describe[A, F[_]: Functor](fa: F[A]) =
implicitly[Functor[F]].map(fa)(a => a.toString)
case class Holder(i: Int)
val jlist: JList[Holder] = {
val l = new java.util.ArrayList[Holder]()
l add Holder(1); l add Holder(2); l add Holder(3)
l
}
val list = describe(jlist) // list: java.util.List[String] = [Holder(1), Holder(2), Holder(3)]
val box2 = describe(Box2(Holder(4), Holder(5)) // box: Box2[String] = Box2(Holder(4),Holder(5))
[/cc]
So, we have a describe function that works for any type that we can map over!
We could also use this with a traditional subtyping approach to have our boxes implement the map method directly with the appropriate signature. This is a little more convoluted, but still possible:
[cc lang=’scala’]
/**
* note we need a recursive definition of F as a subtype of Functor
* because we need to refer to it in the return type of map(…)
*/
trait Functor[A, F[_] B): F[B]
}
case class Box[A](a: A) extends Functor[A, Box] {
def map[B](f: A => B) =
Box(f(a))
}
def describe[A, F[A] a.toString)
val box = describe(Box(Holder(6))) // box: Box[String] = Box(Holder(6))
[/cc]
As a bonus, this last example quite nicely shows how subtype polymorphism is strictly less powerful and also more complicated (both syntactically and semantically) than ad-hoc polymorphism via type-classes.
Postscript
These techniques can lead to some very general and powerful libraries, such as scalaz, spire and shapeless. These libraries may take some getting used to, and as many of these generalizations are inspired by the mother of all generalizations – mathematics – they have names that need learning (like Monad).
However, the techniques are useful without needing to use scalaz. HKT is important for creating type-classes, and creating your own type-classes to encapsulate things like JSON encoding may be of value to your project. There are many ways this can be used within Scala.
If you’re interested in reading more, here’s the original paper for Scala. Among other things, it contains the following very useful graphic:
Also note that the Scala 2.11 REPL is getting a :kind command although its output is a little more convoluted due to the presence of variance annotations on type parameters.
* Strictly speaking, in Scala List[_] is actually an existential type. For the purposes of this post I am using the [_] notation to show the existence of type parameters. Thanks to Stephen Compall for pointing this out.
** An alternate syntax for a context-bound is an explicit implicit block:
[cc lang=’scala’]
def describe2[A, F[_]](fa: F[A])(implicit functor: Functor[F]) =
functor.map(fa) { _.toString }
[/cc]