Search

Thursday 26 December 2013

Sirius: 1 year retrospective

1 year ago I've started working on the Sirius project. 6 month ago there was the first retrospective where I summarized all achievements and future goals. Generally last 6 years were not the most productive. Nevertheless, there were some achievements even there. So, in this topic I'll summarize again where we are with the Sirius now.

Sunday 27 October 2013

GitHub: integrating project into entire GitHub infrastructure

GitHub provides quite good set of toolsets which can be used to support full development cycle and prepare infrastructure to all necessary output artifacts. Recently, I've described samples how to provide integration with GitHub issue tracker and to wrap it with Maven plugin. So, actually, I have dedicated Maven plugin project which I can use as a separate project. In order to make it as much self-contained as it's needed we should:

  1. Set up new project in the GitHub which includes:
    1. New repository setup
    2. Build script preparation
    3. Release settings
  2. Prepare project documentation generation
  3. Integration with CI
So, at the end, all we have to do is to work with our IDE on adding new code or making some changes to existing modules. Everything else is supported by GitHub infrastructure as soon as we push our changes into remote repository.

Not bad at the beginning. So, let's get started.

Monday 23 September 2013

GitHub: test tracking via Maven plugin

Recently we've organized integration with GitHub by retrieving issues content into the files. All this was done in a form of command line utility. In this post I'll extend the functionality to use this solution as Maven plugin.

Sunday 15 September 2013

GitHub: organizing and automating test tracking with Java

GitHub provides some tracker where we can keep our tasks. The GitHub API provides the ability to access that resources programmatically. Thus, with the help of API I could integrate JBehave and GitHub so that I can run JBehave tests stored in GitHub tracker. But it's not the only case.

With the help of API I can set the correspondence between tests and features. This gives the ability to make the traceability of tests and features they're applied for. Also, what if we use something different from JBehave, e.g. Cucumber, Sprecflow which use flat files for input? In this article I'll describe how to:

  • Organize tests with GitHub tracker
  • Automatically generate traceability matrix
  • Automatically generate feature files based on the content of the test issues

Saturday 8 June 2013

Sirius C#: adding UI Automation library

Recently we've created code interacting with Win32 and Web. Now it's time to expand the coverage to the .NET area. For this purpose we have dedicated library called UI Automation. This library is provided with .NET framework and contains basic API interacting with window objects. At the same time this library can be used for interaction with the standard Win32 controls, however it can be used as an auxiliary modules to expand the coverage of existing library I created before for interacting with Win32 elements. In this article I will create sample control classes and make some demo showing how it works. Also, in this example I'll take the tab control and create simple tests interacting with it. At this point that would be just stand-alone library but in the future it will be integrated into entire Sirius project.

Thursday 16 May 2013

Sirius: half year milestone retrospective

6 month ago I've started working on the Sirius project. A lot of things were done since that time, a lot of things were discovered and a lot of things were changed. In general, a lot of work has been done and more work has to be done. So, the aim of this post is just to make some summary of the work being done so far and further directions definition.

Thursday 9 May 2013

Java: JBehave integration with GitHub

During BDD engines comparison and comparing with Cucumber I've found out really serious advantage of JBehave in comparison to other similar engines. It's the ability to read data from external sources different from text files in specific locations as well as the ability to customize that. And I made some sample JBehave integration with JIRA. I did that because I had a plans for Sirius platform to store all tests in JIRA and read them from that location. Well, that link is still useful for projects which use JIRA as tracking system.

But for my project it still doesn't work as I need a shared JIRA instance so I can read tests from there. I don't have it yet... and then I found I don't need that, actually. Since I store Sirius project on GitHub it would be reasonable to share test related resources on GitHub as well. But it's not just about sharing but reading. Actually, GitHub contains it's own tracker. So, in this article I'll describe how to read JBehave scenarios stored as the tickets in the GitHub issue tracker.

Sunday 28 April 2013

Sirius: ruby package installation testing

All Sirius client components are delivered as installable packages. And for Java and C# they are easy to use just by putting references to appropriate binaries. The Ruby packages are easy to install but it's hard to predict in advance whether they reference properly to all necessary resources. That's because any code becomes accessible via adding reference to the path in the file rather than including package name.

The key difference is that local relative paths work OK only for local sources but when you create Ruby package and install it those paths will be applied relatively to completely different folder. So, if you run local unit tests you don't have guarantee that all the code would work in the same fashion as it would work from the gem location. For this purpose, I've added separate class of tests into Sirius Ruby client packages. This class of tests is called installation tests. The main aim of them is to make sure that all modules are properly accessible after the package is installed. And in general it verifies that packages are operational after installation process.

Sunday 21 April 2013

Sirius: running as windows service

Sirius server part is delivered as a set of Java archives and this part is initiated via command line interface. It brings some inconveniences while running the tests. Thus, before start using client API we should make sure that the server is up and running. Of course, we can do that manually but:

  1. we should be able to do that automatically
  2. when we test client side we shouldn't touch server side at all
So, it's quite convenient to deliver Sirius Server start/stop process to the system. For Windows it can be done by setting server component as the windows service. Since, Sirius Server is just another Java application it requires some additional efforts for making such task however such task appeared multiple times. Here are the related topics on StackOverflow: So, I'm looking for something similar for Sirius Server. In this post I'll describe the settings needed for making Sirius Java application running as the windows service.

Sunday 14 April 2013

Sirius: Adding code style checks

We already set up the build process for Sirius as well as we reserved the placeholders for testing. So, further improvement in this area is to extend build with different tests and checkpoints which make more detail verification. This time we'll add more modules which perform static testing for our components and they should fail the build if static checks didn't pass (especially taking into account that it was initially planned).

What would we look for? Basically that's:

  • Conformity to coding standards - all code standards are made to provide unified approach of writing code as well as reading and maintaining it.
  • Various cases of improper code usage - a lot of actual application errors are results of improper code use or use of potentially dangerous code constructions. So, the earlier we find them the less number of silly mistakes we take during further testing stages
Actually, we've made first approximation for that while we checked the compilation ability. If system isn't compilable it's not working at all. So, we're just making more detailed checks.

Since Sirius modules use different programming languages the tool set will be varying based on that. We'll use the following tool set for that:

LanguageEngine
Java
C#
Ruby
OK. Let's see how it would be added.

Wednesday 20 March 2013

JBehave vs Cucumber JVM comparison

The interest in BDD approach is growing and the more and more different engines appear in that space. However, there're some well known players in that area. We can compare their features however the final choice is still based on the technology used. E.g. if we compare Behat and SpecFlow we definitely can find some gaps and advantages but if we're going to use e.g. C# the Behat is not a subject of interest here. And vice versa.

The more interesting case is when 2 or more engine appear in the same technological area. And it's twice more interesting when these are one of the most popular engines on one of the most popular technology. In this post we'll compare JBehave vs Cucumber against their applicability for Java test automation. We compare them using the same scale as for BDD Engines Comparison post with some additional detalization on features. Additionally, I'd like to mention that we should compare apples to apples so there's no need to dig the Ruby implementation of Cucumber. It's interesting to check the Cucumber Java capabilities against JBehave to make comparison more or less valuable.

All right, we have JBehave, we have Cucumber, so ...

LET THE MORTAL COMBAT COMPARISON BEGIN!!!

Sunday 17 March 2013

Sirius: adding Web client API

We've done some work for Web core functionality. To make the work complete the client API should be added in the same fashion as for Win32 library. At the end of current development there should be an example how to interact with web using Web client.

The following topics will be covered here:

  • General Web client architecture
  • Web client proxy updates
  • Web client class library implementation

Monday 11 March 2013

Web test tools list

Introduction

In this post I'll collect the information about existing web test automation tools (both from vendors and open source). It is represented as the catalogue with basic information rather that some comparison characteristics.

NOTE
The content of this post will be appended/updated from time to time to reflect the most recent changes. Also, some information may be not very precise as it is not so widely accessible. So, any comments related to information corrections are welcome

Sunday 10 March 2013

Sirius: adding Web core functionality

A lot of work were done for Win32 stuff starting from initial core functionality preparation and up to basic operations implementations as well as handling parent/child windows and user input. So, now we can work with Win32 (not in full extand but at least for base operations). One of the further expansion direction is the coverage of the Web functionality. That's the completely new area for Sirius Test Automation Platform and in this post I'll describe the core part creation which mainly affect server side.

Monday 25 February 2013

Ruby: soap4r - fixing wsdl2ruby for 1.9.3 version

Some time before when I was trying to generate SOAP Ruby client for my project I encountered the problem with soap4r library. The key problem was with Ruby 1.8 and 1.9 versions compatibility. I've found several independent fixes for that:

and many others. But there were problems with getting them work properly. So, I had to dig into the installed gem code to find out what causes the errors. In particular, I'm getting error like:
[2011-06-09T17:16:01.012268 #10104] INFO -- app: Creating class definition. I, 
[2011-06-09T17:16:01.012268 #10104] INFO -- app: Creates file 'SessionServic e.rb'. F, 
[2011-06-09T17:16:01.014268 #10104] FATAL -- app: Detected an exception. Stop ping ... undefined method collect' for # (NoMethodError) C:/Ruby192/lib/ruby/gems/1.9.1/gems/soap4r-1.5.8/lib/xsd/codegen/gensupport.rb:2 39:intrim_eol' 
C:/Ruby192/lib/ruby/gems/1.9.1/gems/soap4r-1.5.8/lib/xsd/codegen/gensupport.rb:2 27:in format' 
C:/Ruby192/lib/ruby/gems/1.9.1/gems/soap4r-1.5.8/lib/xsd/codegen/commentdef.rb:2 7:indump_comment' 
C:/Ruby192/lib/ruby/gems/1.9.1/gems/soap4r-1.5.8/lib/xsd/codegen/classdef.rb:51: in dump' 
C:/Ruby192/lib/ruby/gems/1.9.1/gems/soap4r-1.5.8/lib/wsdl/soap/classDefCreator.r b:118:inblock in dump_complextype' 
C:/Ruby192/lib/ruby/gems/1.9.1/gems/soap4r-1.5.8/lib/wsdl/soap/classDefCreator.r b:116
...
Actually I was interested in wsdl2ruby utility which was needed for SOAP client generation. So, what should be done to make it working?

Sunday 24 February 2013

Sirius: Win32 - add Dialog handling and basic input

Recently I've added functionality working with main windows. Next step is to apply functionality to interact with dialog boxes and child controls. It's almost the same except these are child windows and we should carry out additional relationship to main window. Additionally, we should add basic ability to send mouse and keyboard input. Thus, we'll complete basic functionality for Win32. All other code to be written should cover some control specifics rather than common functionality.

Sunday 10 February 2013

Java: JBehave integration with JIRA

During BDD engines comparison I noticed that JBehave supports various data sources for the stories but not just text files. This is quite valuable feature as we can share the stories accross the teams. But what if we share that using existing infrastructure? I mean all the tests are typically stored under some tracking system where they can be easily queried from. Such tracking systems should be the place where the tests are written to and updated at. So, why shouldn't we integrate that with the engine so that tests are designed, stored and modified in the tracking system and the engine like JBehave picks them up from there? That would be the tight integration between tests and their automated implementation. In this post I'll show how to make such integration between JIRA and JBehave.

Saturday 2 February 2013

Sirius: Adding new component Demo

Here is the demo showing how to add new component to the Sirius platform. The example demonstrates how to create server module from the scratch and how to apply client for this.

Wednesday 30 January 2013

Sirius (Java): writing Win32 client library + Live Demo

Recently I described the creation of Win32 server part and at the end I wrote the function which finds the main window by specified attributes and gets the handle for it. This is quite fundamental thing as once we have a handle of the window we can send any messages to it simulating various GUI interaction actions. At the same time the server side library contains only set of functions as Win32 API is all about the set of functions. But it's too raw for writing tests. It's much more convenient to apply some object model to the window objects. So, it requires some effort on the client side. In this article I'll describe the process of Win32 client API creation. I'll create classes interacting with top level windows and create some sample test to demonstrate how that functionality works.

The final output will contain the functionality interacting with top level Win32 windows as well as some demo showing how to work with it.

Thursday 17 January 2013

Java JNA Win32: when default libraries are not enough

I've been working on implementing Win32 interaction functionality using JNA library and encountered the situation when the library became incomplete. The methods of com.sun.jna.platform.win32.User32 interface which is supplied by the platform library contained only subset of the entire functions of user32.dll library. Actually, I've found that while trying to call GetMenu function. It appeared that there's no such function in the Java class supplied with the JNA library. What to do with that? How to make a Java call for functions from existing DLL?

Sunday 13 January 2013

Sirius: modularize the solution

In the previous post I've added another module to support Win32 operations. Actually, there should be a lot of modules to be added. But I put it directly into the Server module. The more components we cover the more modules I have to include. Thus the server code as well as clients become heavy-weight and entire solution becomes huge. But firstly, it breaks one of the ideas of the Sirius platform. It should have an extensible engine which can plug in the modules on fly. Also, it should be adopted to different operating systems (that's why I chose Java as the main language) but what should the Win32 module do, for instance, on Unix? Nothing. So, in such case I don't need to include it. Or even more, I want to make different complectations for Server side depending on what I actually want to use. For this purpose I have to split our components into functional modules and provide the ability to configure what modules I should include. Also, I would be useful to provide the ability to deliver either entire solution or some parts of it. In this post I'll describe the steps for doing this.

Thursday 10 January 2013

Sirius: core Win32 functionality support

Once we've done with infrastructure it's time to work on development itself. Now I'll start working on the functionality performing interaction with Win32 objects. In this article I'll describe basic preparations for Win32 interactions as well as I'll describe the core functionality to capture the window on screen and perform some manipulations with it. And of course this activity is done in the scope of Sirius development. So, it would be bound to the entire architecture of the platform.

Wednesday 2 January 2013

Sirius: Testing setup

Since Sirius is going to be used for testing purposes it's not polite to leave it without testing. Major specifics of this solution is that different parts of entire system are written using different programming languages. At the same time all those parts reflect the same functionality. It means that the tests would be the same for all clients. Well, they would be the same for server part as well (at least most of the core tests). It's not very nice to copy/paste them and then migrate to different languages. So, obviously there's going to be a lot of duplicating work. So, I have to do something to minimize that duplication as well as minimize the maintenance effort in case I want to change some tests. It's valid especially for client side as core part doesn't have any local specific logic. All immediate client methods are actually callers for the same server side methods.

So, there's going to be multiple languages but the same interface for tests. It's not trivial solution but I found that.