Wednesday, February 29, 2012

Scala and Selenium

Update: I'm now using fluentlenium instead of this mechanim with the play framework.  It's much better!

I've been working with Scala for awhile, and I've just dipped my toe into Selenium.  Selenium is a web testing tool that allows you to drive a browser like Firefox, to do more detailed testing of web pages.  Coupled with Cucumber, doing BDD based testing is pretty cool.

Setting this up in Java is a fair bit of work, and I managed to get it going with a little help from the internet.  Then I wondered what it would look like in Scala.  He below lies a configuration for settings up Selenium testing using Cucumber in Scala.

You can find all the pieces for this out there, but I hope having them all in one happy place will save a few folks some time! I could have used sbt, but chances are, if you're using sbt, you can easily figure out how to convert this pom. I use maven here because it's a common denominator (perhaps the lowest, but still), you can import it easily into IntelliJ or Eclipse or Netbeans.

As per usual, I'm no Scala expert, but in my project this works though there may be more idiomatic ways of solving some of the things in here. Please please comment if you have thoughts or ideas on this. The pom file:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.example</groupId>
    <artifactId>selenium-tests-scala</artifactId>
    <version>1.0.0</version>
    <inceptionYear>2012</inceptionYear>
    <packaging>jar</packaging>
    <properties>
        <scala.version>2.9.1</scala.version>
    </properties>

    <repositories>
        <repository>
            <id>scala-tools.org</id>
            <name>Scala-Tools Maven2 Repository</name>
            <url>http://scala-tools.org/repo-releases</url>
        </repository>
        <repository>
            <id>sonatype-snapshots</id>
            <url>https://oss.sonatype.org/content/repositories/snapshots</url>
        </repository>
    </repositories>

    <pluginRepositories>
        <pluginRepository>
            <id>scala-tools.org</id>
            <name>Scala-Tools Maven2 Repository</name>
            <url>http://scala-tools.org/repo-releases</url>
        </pluginRepository>
    </pluginRepositories>

    <dependencies>
        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-library</artifactId>
            <version>${scala.version}</version>
        </dependency>
        <dependency>
            <groupId>org.scala-lang</groupId>
            <artifactId>scala-compiler</artifactId>
            <version>${scala.version}</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.10</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.specs</groupId>
            <artifactId>specs</artifactId>
            <version>1.2.5</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>2.17.0</version>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-junit</artifactId>
            <version>1.0.0.RC16</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>info.cukes</groupId>
            <artifactId>cucumber-scala</artifactId>
            <version>1.0.0.RC16</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <sourceDirectory>src/main/scala</sourceDirectory>
        <testSourceDirectory>src/test/scala</testSourceDirectory>
        <plugins>
            <plugin>
                <groupId>org.scala-tools</groupId>
                <artifactId>maven-scala-plugin</artifactId>
                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                            <goal>testCompile</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <scalaVersion>${scala.version}</scalaVersion>
                    <args>
                        <arg>-target:jvm-1.5</arg>
                    </args>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-eclipse-plugin</artifactId>
                <configuration>
                    <downloadSources>true</downloadSources>
                    <buildcommands>
                        <buildcommand>ch.epfl.lamp.sdt.core.scalabuilder</buildcommand>
                    </buildcommands>
                    <additionalProjectnatures>
                        <projectnature>ch.epfl.lamp.sdt.core.scalanature</projectnature>
                    </additionalProjectnatures>
                    <classpathContainers>
                        <classpathContainer>org.eclipse.jdt.launching.JRE_CONTAINER</classpathContainer>
                        <classpathContainer>ch.epfl.lamp.sdt.launching.SCALA_CONTAINER</classpathContainer>
                    </classpathContainers>
                </configuration>
            </plugin>

        </plugins>
    </build>
    <reporting>
        <plugins>
            <plugin>
                <groupId>org.scala-tools</groupId>
                <artifactId>maven-scala-plugin</artifactId>
                <configuration>
                    <scalaVersion>${scala.version}</scalaVersion>
                </configuration>
            </plugin>
        </plugins>
    </reporting>
</project>


The features file is used to speak BDD language. This file is processed by Cucumber and matched against the step definitions file.
Feature: The home page should show useful content

Scenario: Load the home page

     Given I want to see my home page
     Then I should see valid content


This class functions essentially as a marker to JUnit to execute Cucumber to build the test instead of scanning this class for methods directly. You'd think there'd be an easier way to do this without having to have something that amounts to no more than a stub here.
package com.example.selenium.cucumber

import cucumber.junit.Cucumber
import cucumber.junit.Feature
import org.junit.runner.RunWith

/**
 * Test definition class that indicates this test is to be run using Cucumber and with the given feature definition.
 * The feature definition is normally put in src/main/resources
 */
@RunWith(classOf[Cucumber])
@Feature("Home.feature")
class HomePageTest {
}


The step definitions file is used by Cucumber to match the features text agains. There are several basic predicates in Cucumber like "When" and "Then" and "And" that you can use via the DSL. Cucumber supports many languages in this regard so you're not limited to English for this.
package com.example.selenium.cucumber

import org.openqa.selenium.WebDriver
import junit.framework.Assert.assertEquals
import junit.framework.Assert.assertTrue
import com.example.selenium.pageobject.HomePage
import com.example.selenium.WebDriverFactory
import cucumber.runtime.{EN, ScalaDsl}

/**
 * Step definitions for the Home Page behaviours
 */
class HomeStepDefinitions extends ScalaDsl with EN {
    Before() {
        driver = new FirefoxDriver()
    }

    After() {
        driver.close()
    }

    Given("^I want to see my home page$") {
        home = new HomePage(driver)
    }

    Then("^I should see valid content$") {
        var actualHeadLine: String = home.getTitle
        assertEquals(actualHeadLine, "My Awesome Home Page!")
        assertTrue("Page content should be longer than this", home.getContent.length > 4096)
    }

    private var driver: WebDriver = null
    private var home: HomePage = null
}

The HomePage object here is used to store information about the home page, and potentially actions that you can perform whilst on the HomePage which I've omitted here for simplicity. In my working system, I make most of this live in a base class that all my page classes extend. Things like the ability to screen shot are useful everywhere, as is reading the title and page content. Some generic tests that can simply match the title and content for a whole bunch of pages can be very useful for bootstrapping the test process. They might not be good test, but they're definitely better than nothing!
package com.example.selenium.pageobject

import org.openqa.selenium.By
import org.openqa.selenium.WebDriver
import org.openqa.selenium.WebElement

/**
 * A page object used to represent the home page and associated functionality.
 */
class HomePage(driver: WebDriver, baseUrl: String) {

    private var driver: WebDriver = null
    private var baseUrl: String = null

    def getDriver: WebDriver = driver

    protected def takeScreenShot(e: RuntimeException, fileName: String): Throwable = {
        FileUtils.copyFile(
            (driver.asInstanceOf[TakesScreenshot]).getScreenshotAs(OutputType.FILE),
            new File(fileName + ".png")
        )
        e
    }

    def getContent: String = getDriver.getPageSource

    def getTitle: String = getDriver.findElement(By.tagName("title")).getText
}

I hope I haven't left anything critical out that would be required to get this system bootstrapped. Obviously this set up doesn't do very much, but, it will at least get you up and running. You can probably figure out the rest from the Cucumber and Selenium documentation at this point. I might post a follow up with some more detail, but it took me a few weeks to get this posted!

6 comments:

  1. You can also do some Given/When/Then specifications with specs2:

    http://etorreborre.github.com/specs2/guide/org.specs2.guide.SpecStructure.html#Given+%2F+When+%2F+Then

    The main differences with cumcumber are:

    - a bit more typechecking with the type of results which go from Given to When to Then

    - the ability to execute Then verifications concurrently

    - the ability to generalize the examples with some ScalaCheck goodness

    ReplyDelete
    Replies
    1. I had looked at this, but I find the syntax doesn't meet my needs. It's not as simple and straightforward as a cucumber feature file which allows me to make a test feature entirely legible and comprehendible to a business stakeholder. It's entirely separate from the code that runs it (at least superficially). The G/W/T pattern matching is fairly primitive and clunky too. I can build tests with the cucumber mechanism that are fairly sophisticated and also very DRY without much effort.

      In general, I like spec tests a great deal. I find for general unit tests I'll take spec tests any day of the week. I think web page testing isn't your average test situation though.

      My one complaint is that chaining test features together seems problematic, I'm still trying to figure out the best way to do it.

      Delete
  2. I think you would be much better of with Geb (combined with Spock), rather than fiddling around with Scala. Have a look at this sample:

    https://github.com/geb/geb-example-gradle

    ReplyDelete
    Replies
    1. Geb does look interesting. Seems a bit cleaner than what I'm doing right now. It's a pity it uses Groovy rather than Scala, but I'm not entirely opposed to Groovy if it's good for the usage, and in this case, it does look good. The Geb documentation is pretty easy to navigation too, and that's something that's pretty important. Selenium's documentation is kind of a pain, as is Cucumbers, which is one reason I wrote this.

      Delete
    2. Man I really dislike GEB. What a pain. At my previous job it took us nearly a year to get that thing going and we still have non deterministic results. Cucumber was up and running in 15min and tests were being written. The heirarchy at my previous place loved Groovy, so they refused Cucumber on the spot.

      Geb's main pain points for me, was it's fragility being tightly coupled with specific versions (Groovy 1.7, Spock 0.65 or whatever.) Then you get a problem with running FF tests with the latest FF version, so I upgrade webdriver to find it isn't compatible with groovy 1.7 or spock 0.65 ugh!

      then when you get crazy errors every day, you google for solutions to find... none. So you're own, with a turn around time of yesterday... man. what a pain it was.

      Delete
  3. Another tidbit of information for you:
    ScalaTest has BDD features built-in with
    FunSpec (Scala Trait inspired by RSpec) http://www.scalatest.org/scaladoc/1.7.2/org/scalatest/FunSpec.html
    and Given/WhenThen
    Reporter will create a specification-like document.
    The difference like you mentioned is Cucumber will keep humanly-readable
    feature files de-coupled from code and ScalaTest keeps that in code and relies on reporters.

    See examples here:
    https://scalatest.org/user_guide/tests_as_specifications

    ReplyDelete