Scala is very nice - very very nice

Today I am gushing over Scala's par method and XML literals. I am fetching about 30,000 entries over REST calls. The server isn't super fast on this one, so each call takes a bit of time. Enter list.par stage left.

list.par creates a parallelizable list which given an operation will perform it in parallel across multiple CPUs.  It spawns threads and performs the operation, then joins all the results together at the end, very handy.

This little three letter method is turning what would be a very very long arduous process into a much less long one. Much much less.

val myList = io.Source.fromFile("list.txt").getLines.par.map { x =>
  callService("FooService", "{id=\""+x"\"}")
}

It gets better. In Scala, XML can be declared as a literal. Not only that, but it runs inline like a normal literal, with a few special rules. This service is combining a bunch of json into an XML output.

val myOutput = io.Source.fromFile("list.txt").getLines.par.map { x =>
  callService("FooService", "{id=\""+x"\"}")
}.map { x =>
  Json.parse[Map[String, Object]](x)("url").toString
}.map { x =>
  <entry>
    <url>{ x }</url>
  </entry>
}.toString

Which I can now happily write to wherever I need to, a file, or a web service response. Nifty in the extreme.

In 2012, we live in a world of JSON and XML. Perl had it's day when text processing was king. Today, a language is needed that can cope with JSON, XML and Parallelization and still yield sane-looking code. I'm not a big Ruby fan, as anyone who knows me will tell you, but I'm willing to keep an open in. I'd like to see if Ruby can do this kind of thing as elegantly and easily and demonstrate it's a language for the web in 2012.  Also, I should mention Akka as well, though I don't yet know enough about it, other than it can allegedly take parallelization inter-compuer with similar simplicity.

Simple Scala scripts : Scan a directory recursively

I'm using Scala increasingly as a scripting language at the moment. As my confidence with it is increasing, I'm finding it's becoming more and more useful for those throw-away scripting situations.  Especially when then end up being not so throw-away after all.

def findFiles(path: File, fileFilter: PartialFunction[File, Boolean] = {case _ => false}): List[File] = {
  (path :: path.listFiles.toList.filter {
    _.isDirectory
  }.flatMap {
    findFiles(_)
  }).filter(fileFilter.isDefinedAt(_))
}

(replace {} with (), ditch newlines and it goes on one line well-enough, just doesn't fit in a Blogger template that way)
We might be duplicating the a shell find:

find | grep 'foo'

or

find ./ -name "foo"

And whilst the Scala is more complex, the Scala function can do operations on a File object, which gives you a lot of the rest of the power of the find command thrown in to the bargain. Plus, as it accepts a partial function, you can chain together filters. If you truly just wanted an analog for find:

def findFiles(path: File): List[File]  = 
  path :: path.listFiles.filter {
    _.isDirectory
  }.toList.flatMap {
    findFiles(_)
  }

Which is less complex that the first. This is still more work than find, but, the list you get back is an actual list. If you added anything useful to your find, say an md5 for each file, it gets less happy

find ./ | awk '{print "\""$0"\""}' | xargs -n1 md5sum

Maybe there's a better way, but that's what I've always ended up doing. The Scala is starting to compete now. Bump up the complexity one more notch, and I think Scala actually starts becoming less code and less obscure.

You might also notice that the example above can be fit nicely within the Map/Reduce paradigm. Scripting that is not only relatively easy, but can also be thrown at Hadoop for extra pzazz, and NoSQL buzz-worthyness.

First day with Play

I started looking at play 2.0 for the first time today.  I got a few hours with it at least, and I've been impressed with a few things.

The first and biggest thing is perhaps the most simple: compile on the fly.  Grails does this, but very badly, and if I'm observing right, I think I can see why.  It seems that Play compile checks at request time, not at file save time.  As one who grew up computing, Ctrl-S or it's equivalent has become a nervous tick.  Grails recompiling at save time almost always just ends up trashing the environment as I save about ten times a minute, and I end up having to restart it, which is very slow.

With FSC, Play compiles changes very quickly and I barely notice a lag at all.  It doesn't get stuck anywhere I've noticed yet either like Grails can.

I feel like within a couple of hours, I got pretty far into having a functional, albeit basic web app going.  Working with Anorm is interesting too, I'm not sure if I like how they've done it yet, but after years living with JPA and Hibernate's arcane query structures and unexpected query stupidity (although there was always a good reason, it was still annoying), I find this way of integrating SQL and code better than most.  It has some similarity with O/R Broker which is what I've been using with Jersey so far, but Anorm is more sophisticated and I think easier to work with.

The error reporting in Play is also excellent.  You can see quickly and precisely what went wrong with your code, there's no guesswork and decrypting enigmatic error messages, it just tells you with your source code right there: this is where it's broken!

OMG <3 Scala

Today was a good day. I wrote a somewhat complex piece of logic for a jump planner for EVE online, and, because of the awesomeness of Scala, it worked as conceived, first time.

This is one of the many reasons I love functional programming. As you take your problem and reduce it down to it's most basic components, strip out logic, simplify to a mathematical expression, simplify that expression, suddenly errors become clear in the very writing of it.

Scala feels so much closer to that expression than Java. So much verbosity is inherently prone to error, writing error free code in Scala seems dramatically more possible than in Java.

SBT and Gradle

Liistening to a not quite cent episode of Java Posse podcast, I heard them talk about two new build tools that are in the running to succeed Maven and Apache Ivy. They are SBT and Gradle respectively.

As with the rest of the Java community who are moving on, one is Scala based and the other is Groovy based.

I'm not going to get into a Scala vs Groovy thing here, as i can't say I know either of them to a high enough level to really make a fair comparison.

I've had a poke at both of them, and so far, I'm really diggin the simplicity of SBT and it's interactive features for working with Scala, which are super nice.

I've only got a small example going with my EVE project, and it so small it's barely worth posting. Having said that, it did clue me in to working with scalatest, which is much more elegant than JUnit. I'm curious to use it also with JUnit and java code, I've heard that it can work well with both Scala sources and Java. The interactive test suite features seem like a really awesome feature, continuous integratinon built right in.