Thursday, November 29, 2018

Future[T] and Future[Try[T]] - Round Two!

In my last post, I was discussing some of the issues I'm facing around Future[T]; and how if you look inside the implementation of Future, what you see is

val value: Option[Try[T]].

Of course, for my part, I want access to that Juicy Try[T].  without it, flows can just fail; another point in case, the post-process code in our system:

for {
  flowDone <- flow.runWith(Sink.foreach(e => logger.trace("Updated reference " + e)))
  catDone <- categoryService.updateCategoryListWithProductCount(loginId)
  childSkus <- backPropagateChildSkus(loginId)
  reIndexDone <- searchReindexerService.callReIndex(loginId)(ws)
} yield reIndexDone

Once the product ingest work is done, I then want to chain that future through a series including updating category classifications, updating Child Sku information and sending the whole lot off wholesale for reindexing in the search cluster.

With the current implementation and function of Future, even if these methods all return Future[Try[T]], any one of them might fail for an unpredictable reason, and generate a Failure[Try[T]] where the Failure's Try is Failure, and the inner value is never exposed.  And in fact, on the first execution this is precisely what happened.  Another exception was swallowed I suspect, because whilst all the products seem to have been inserted, the reindex started as indicated on logging, but didn't complete successfully.

Something must be done!

Attempts to do something very brute force like subvert Future[T] to return Future[Try[T]] in all cases seem dubious; particularly as we now have the problem of dealing with somehow making sure Future[Try[T]] still returns Future[Try[T]] and not Future[Try[Try[T]].  My type foo is not strong enough to untangle that one; and perhaps that's a good thing.

So having arrived at that conclusion, what do I want to do about it.  I think I'm going to give this a spin:

class FutureTryAutoTransformer[T](original: Future[Try[T]])(implicit executionContext: ExecutionContext) {
  val liftF: Try[Try[T]] => Try[Try[T]] = {
    case Success(Success(v)) => Success(Success(v))
    case Success(Failure(e)) => Success(Failure(e))
    case Failure(e) => Success(Failure(e))
  }

  def lift: Future[Try[T]] = original.transform[Try[T]](liftF)(executionContext)
}

implicit def future2FutureTryAutoTransformer[T](
  f: Future[Try[T]])(implicit executionContext: ExecutionContext) = 
   new FutureTryAutoTransformer[T](f)

With that little piece of magic, I have a way to "lift" the Future[Try[Try[T]] up so that the inner and out Try components are handled together and I get a real Future[Try[Try[T]] with no hidden agendas!!

let's see how we feel about that - I've heard tell of a scalaz solution which I might look in to... watch this space!

Tuesday, November 27, 2018

On Future[T] and Future[Try[T]]

In the Scala universe there is some debate of the usage of Future[Try[T]], and how to best encapsulate failure in the context of a Future.  For my part, I like using Monads to communicate context, and meaning of the expectation, especially around failure.  One of the biggest reasons for the existence of Option[T] is to pro-actively handle null cases; and with Try[T], the same thing with exceptions (noting that Try is not technically Monadic, but... it's close).  This becomes especially bothering once you drop into an Akka streams situation with flows where errors can easily just get eaten by the system completely, with no exception trace or notification.  I have one application where it ingests millions of rows, and occasionally the flow blows up, and what do you see on the logging?  Nothing at all.

So - how can you at least address this situation; if you're like me, and like explicit understanding based on your Type arrows; how can you wrap this up in a way that gives you the context you desire:


  def tryingFutTry[T](f: => Future[Try[T]])(implicit executionContext: ExecutionContext): Future[Try[T]] = 
    try {
      f.recoverWith({ case e: Throwable => Future(Failure(e))})
    }
    catch {
      case e: Exception => Future(Failure(e))
    }

  def tryingFut[T](f: => Future[T])(implicit executionContext: ExecutionContext): Future[Try[T]] = 
    try {
      f.map[Try[T]](Success.apply).recoverWith({ case e: Throwable => Future(Failure(e))})
    }
    catch {
      case e: Exception => Future(Failure(e))
    }

  def trying[T](f: => T)(implicit executionContext: ExecutionContext): Try[T] = try {
    Success(f)
  }
  catch {
    case e: Exception => Failure(e)
  }
Whilst this isn't very pretty, or perhaps even very well named; I'm not loving it yet; it does at least give you a way to "lift" non-try wrapped Future in a Try context to lift out the failure case from inside the Future that gets eaten, and allow you to expose swallowed exceptions when you have an explicit Try context that may not be catching all exceptions.

A lot of these ideas were take from a blog post that I found here:
https://alvinalexander.com/scala/how-exceptions-work-scala-futures-oncomplete-failure