Make Junit not stop when assertion fails

Let’s say we have multiple assertion check in one test method, Junit will stop at the first assertion error.

String str = null;
assertNotNull("String str should not be null", str);
str = "";
assertEquals("String str should have the lenght of 1", str.length(), 1);  

I am using Junit class to do WebDriver test. I want to all assertions are executed no matter there is error or not. Junit Error Collector Rule allows me do that.

@Rule
public ErrorCollector errorCollector= new ErrorCollector();

@Test
public doSomeTest() {
    //do something...
    errorCollector.checkThat("penalty", is(savedPage.getPenalty().getValue()));
    errorCollector.checkThat("exclusion", is(savedPage.getExclusion().getValue()));
}

The code above will run all the check no matter there is error or not

Junit 4 Parameterized Test

I am using WebDriver to do the automatic web test. Different web driver instance can start different browser to do the test.

For example, this code will start firefox.

WebDriver driver = new FirefoxDriver();

This code will start internet explore to do the test.

WebDriver driver = new InternetExplorerDriver();

I want to run my test on both firefox and IE to make sure my java script works on both. The test is the same, the only difference is to use different driver instance. I need run one test twice, one with firefox driver, the other one with ie driver. Junit parameterized is exact for that purpose.

I put this code in the abstract super class of all my web driver tests.

private static WebDriver firefoxDriver = new FirefoxDriver();
	private static WebDriver internetExplorerDriver = new InternetExplorerDriver();
	
	@Parameters
	public static Collection data() {
		return Arrays.asList(new Object[][]{{firefoxDriver}, {internetExplorerDriver}});
	}

My web driver test also need provide a constructor using WebDriver as argument.

public (final WebDriver driver){
this.driver = driver;
}

In the example above, junit will pass “firefoxDriver” to the test and run all the tests and then pass “internetExploreDriver” and test again.

Do web auto test with maven integration testing and Selenium RC

Selenium RC allows us to use different language (java, ruby, C, groovy…) to build the test case for web auto testing.  It is better than doing selenium test by html script because in code like java, we can reuse the code.

        @Before
	public void setUp() throws Exception {
		selenium = new DefaultSelenium("localhost", 4444, 
				"*chrome", "http://localhost:7001/");
		selenium.start();
	}
	
	@Override
	@After
	public void tearDown() throws Exception {
		super.tearDown();
		selenium.stop();
	}
	
	@Test
	public void testSite() throws Exception {
		loadPageForManaging();
		verifyPageEditableForCreatingNewSite();
		Site site = createSiteObject();
		typeSiteToUIAndSave(site);
		verifySite(site);
		updateSiteObject(site);
		typeSiteToUIAndSave(site);
		verifySite(site);
		......
        }

	protected void typeSiteToUIAndSave(final Site site) {
               // add new site
               selenium.type("site.code", site.getCode().toString());
               selenium.type("site.name", site.getName());
               ...
        }

        protected void verifySite(final Site site) {
		verifyEquals(String.valueOf(site.getCode()), selenium.getValue("site.code"));
		verifyEquals(site.getName(), selenium.getValue("site.name"));
                ...
        }

From the code, you can see typeSiteToUIAndSave method and verifySite was reused a couple of times.

The selenium RC testing need two things:

1. selenium RC server is running. Maven selenium plugin is able to do that.



2. the application to be tested is running.

I want to use maven to build the package, then deploy, then run the integration testing by maven. The “deploy” here is not the “deploy” in maven life cycle, as in maven life cycle, “deploy” happens after “integration-test”. We implement the above testing in junit class. The first step “package” will run junit test. Obviously we don’t want the selenium test to be executed before our deploy. To avoid that, we make selenium test class name end with “IT” (integration test) and use the maven-failsafe-plugin to run the integration test. The maven-failsafe-plugin will default look up the classes with name”*IT” and execute them.

            <plugin> 

                <artifactId>maven-failsafe-plugin</artifactId> 

                <version>2.6</version>   
                <executions> 

                    <execution>                         
                        <goals> 

                            <goal>integration-test</goal> 

                        </goals> 

                    </execution> 

                </executions> 

            </plugin>

Set up maven selenium plugin to start up the RC server before the selenium test is executed

           <plugin>                 
                <groupId>org.codehaus.mojo</groupId>  
                <artifactId>selenium-maven-plugin</artifactId>  
                <version>1.1</version>  
                <executions>  
                    <execution>  
                        <phase>pre-integration-test</phase>  
                        <goals>  
                            <goal>start-server</goal>  
                        </goals>  
                        <configuration>  
                            <background>true</background>  
                            <logOutput>true</logOutput>  
                        </configuration>  
                    </execution>  
                </executions>  
            </plugin>

I also setup a sql script to be executed after the selenium test to remove the testing data from the database

            <plugin>  
                <groupId>org.codehaus.mojo</groupId>  
                <artifactId>sql-maven-plugin</artifactId>  
                <version>1.4</version>  
                <dependencies>  
                    <!-- specify the dependent JDBC driver here -->  
                    <dependency>  
                        <groupId>com.oracle</groupId>  
                        <artifactId>ojdbc14</artifactId>  
                        <version>10.2.0.1.0</version>  
                    </dependency>  
                </dependencies>  
                <!-- common configuration shared by all executions -->  
                <configuration>  
                    <driver>${dbDriver}</driver>  
                    <url>${dbUrl}</url>  
                    <username>${dbUserName}</username>  
                    <password>${dbPassword}</password>  
                </configuration>  
                <executions>  
                    <execution>  
                        <id>cleanData</id>  
                        <phase>post-integration-test</phase>  
                        <goals>  
                            <goal>execute</goal>  
                        </goals>  
                        <configuration>  
                            <srcFiles>  
                                <srcFile>dbscript/cleanSeleniumTestData.sql</srcFile>  
                            </srcFiles>  
                        </configuration>  
                    </execution>  
                </executions>  
            </plugin>

Unit testing, regression testing and integration testing

Regression Testing

Any time you modify an implementation within a program, you should also do regression testing. You can do so by rerunning existing tests against the modified code to determine whether the changes break anything that worked prior to the change and by writing new tests where necessary.