KSP and Me

I’ve been using Kotlin Symbol Processing (KSP) for a few years so I thought I’d reflect on how I like to work with it to stay productive.

First things First

Let’s start by recognising if you are new to KSP it is hard to get up to speed, it’s not impossible but it will require some graft to really get stuck in. Many of the blog posts I read when I was starting were very good at helping you get something compiling but then pretty much finished there. Without someone holding my hand or giving me cues of where to look I was kind of stuck not really knowing the potential of the tool I was learning.

Don’t treat what you read on the internet as gospel

Many of the blog posts I read when starting out had a similar pattern of suggesting you should use the visitor pattern and KotlinPoet, without really saying why you’d want to use them. I’ve read the Gang of Four book many moons ago but had all but forgotten the visitor pattern and I’d never heard of KotlinPoet so that’s two things I was expected to learn just to follow an introductory tutorial.

Thankfully I’m a few years in and I’ve mostly managed to avoid using the visitor pattern for my use cases. My coding style these days leans more towards a functional style so less common OO patterns just feel alien and slow me down.

For example to get all of a class’ nested children I could use the visitor pattern something like this:

class MyVisitor: KSDefaultVisitor<Unit, Klass?>() {
    override fun visitClassDeclaration(classDeclaration: KSClassDeclaration, data: Unit) = Klass(
        classDeclaration.simpleName.asString(),
        classDeclaration.declarations.mapNotNull { it.accept(this, Unit) }.toList()
    )

    override fun defaultHandler(node: KSNode, data: Unit): Klass? = null
}

Tangent: None of the examples I read at the time actually accumulated results in this functional style using the second type parameter but instead opted but having an instance variable that you accessed after parsing was completed.

Or I could use a more functional style like this:

fun KSClassDeclaration.nestedClasses(): Klass = Klass(
    simpleName.asString(),
    declarations.filterIsInstance<KSClassDeclaration>().map(KSClassDeclaration::nestedClasses).toList()
)

The functional style I personally find more direct and I can see the recursion happening I’m not relying on learning what the various conformances to visitor are and which is right for my use case and the methods I need to use/implement (accept, defaultHandler) and why.

Anyway I’m not trying to sell one approach over the other because that’s for you and your team to thrash out. I’m mostly just saying if it works then use it, you don’t have to feel like I did that I was somehow holding it wrong because my code didn’t look like all the blog posts I was reading.

The other good thing to report is that I haven’t needed to learn KotlinPoet, again for the things I’ve worked on multiline string literals have been more than adequate. I mean I know what the Kotlin code I want to generate should look like so having an extra layer in the middle doesn’t add much for me personally.

Separate parsing and generating

When I started I kept trying to build up the final String of what the Kotlin source code should be whilst parsing the code. This is not a great idea as you soon tie yourself in knots. What compilers tend to do, which is the pattern I follow now is

• Parse the input into some intermediate representation
• Process the intermediate representation
• Render the output

For step 1 I like to take the types provided by KSP such as KSClassDeclaration and extract out the information I need into simple data class types. That way the processing logic I write next doesn’t need to know about KSP and the task is more focussed on gathering all the data that my processor thinks is interesting.

Once I have the data I’ll then do any validation, filtering or mapping to new representations. At this point I’m working with simple immutable data classs with well named properties, which is much preferred to having all my business logic calling all combinations of deeply nested resolve(), declaration, asString(), etc.

The final step is rendering, which is very often now just a collection of string templates that interpolate in the nicely structured data from the previous step.

I think there are a few great advantages to separating things out:

• You can generate different code for different targets (e.g. Kotlin/JS vs Kotlin/JVM) in a much simpler way
• Future readers don’t have to follow a potential mess of building a string whilst parsing
• More easily add unit tests around the business rules in the processor

Validate before you generate

Linked to the mistake mentioned in the section above about trying to do things in one go I would fully recommend writing out the code you want to generate manually and checking it works. I’ve found that I was constantly starting off with a simple picture of what I needed to generate and it seemed so obvious what was needed that I started writing the generation code. The issue is I’m not a very good developer and the simple code I imagine never really works and often requires changes. It’s much simpler to edit, compile and run code directly rather than trying to change the code to generate new code so that you can run and validate it.

Example use cases

The biggest pain point for me was not having that spark of inspiration for what I could be doing with KSP. Here’s a few things that me and my team have used KSP for:

• Validation tasks
  • Ensuring that a module correctly uses functions or computed properties, this is a bit niche but this module houses UI strings and if we use properties then every single string would end up in the JS artifact even if it wasn’t referenced. To avoid every JS artifact having every possible string we rely on the dead code elimination you get when the compiler notices you don’t invoke a function.
  • Ensuring that certain types conform to Serializable to support Android. If we forget the conformance then we could crash at runtime if an activity tries to serialize state.
• Generate type aliases
  • KMP doesn’t export typealises to iOS and the naming rules for types can be a little funky. Some times subtypes have dot separators (Parent.Child) and other times the symbol names are just smashed together (ParentChild). This is super confusing and we want to alias the most recent versions of some generated types so iOS developers never know about the actual versioning. The processor for this outputs Swift code, which is then packaged via SKIE.
• Generate type safe routing helpers
  • A colleague wrote a processor that will read various spring boot annotations to calculate the path for an endpoint and what arguments are required. This is then all used to generate typesafe helper functions that allow people to do routing in a much safer manor.
• Generate DSL versions
  • Me and a colleague wrote a pretty comprehensive processor that generates type safe versioned DSLs allowing us to migrate away from a system that meant adding a new version of our DSL required fairly specialised knowledge of our versioning system + many hours-days of work and resulted in inconsistent results to now mostly just bumping a number.
• Generating observability wrappers
  • Me and various colleagues wrote a processor that takes a class with some annotations sprinkled on it and generates a wrapper class that knows when properties are being written and will require us to recompute a new state. This processor also generates type safe bindings that allow us to bind our UI to these properties.
• Generate per request caches in webflux
  • I’m still learning webflux but a requirement came up to have a per request cache, this would normally be done with an @Cacheable annotation on a method in a normal thread based spring boot application. What I ended up spiking was having KSP look for an annotation of @Memoize which then generated a CoWebFilter to create a typed caffeine cache and slap it in the coroutine context. Then the KSP would generate a wrapper class that delegates to the original after trying the cache. This generated delegate wrapper would have a @Primary annotation so spring would wire it in rather than original.

There’s plenty more example uses out there these days if you look around but all of the above are either in live active projects or hopefully will be soon.

Conclusion

I think it’s great to have good documentation on how to use a library but sometimes the thing that is missing is the little bit of inspiration that get you thinking about how you could apply a technology to your project. I’m glad we embraced KSP and we have done away with so much boilerplate code and all the opportunities for mistakes and inconsistencies to sneak in that makes maintenance harder.