Q7

Verify that runner starts crashed AUT properly

Details

  • Type: Task Task
  • Status: Resolved Resolved
  • Priority: Major Major
  • Resolution: Fixed
  • Affects Version/s: 1.3.13
  • Fix Version/s: 1.3.14
  • Component/s: Runner
  • Labels:
    None
  • Test Mode:
    Manual

Description

Sometimes we get reports from customers that some tests are failing with a message "connection is not available" (e.g. http://support.xored.com/helpdesk/tickets/1120). We need to make sure that runner restarts AUT properly on time, i.e. if AUT terminates during a test case, it should startup before the next test. I suggest to perform a manual validation – create a small test suite with tests like this:

  1. Passing test case
  2. Test case, which causes application shutdown (i.e. just executes shutdown-aut command or clicks File -> Exit menu)
  3. Failing test case
  4. Passing test case

and launch it with runner
Resulting report should contain exactly 2 failures - for a 2nd and 3rd tests. If it is not the case, create a bug

Activity

Hide
Ulyana Skorokhodova added a comment - 29/Aug/14 4:21 PM

Verified.
Made a corresponding test suite (added one additional test case with File/Restart).
The Runner's behaved as expected.
Here is a full log:

Pass 1 (5) processed. 0 failed. spent: 0:06, 0:06 mins remaining. 1_Passing Test Case. time: 5269ms 
Pass 2 (5) processed. 0 failed. spent: 0:29, 0:14 mins remaining. 22_Test Case with File-Restart. time: 22719ms 
AUT-0:Restarting (restartId=1)
AUT-0:Product: org.eclipse.rcptt.platform.product
AUT-0:Application: org.eclipse.ui.ide.workbench
AUT-0:Architecture: x86_64
64bit arch is selected because AUT uses launcher library
	"../../../plugins/org.eclipse.equinox.launcher.cocoa.macosx.x86_64_1.1.101.v20120109-1504" specified in config file: rcptt.app/Contents/MacOS/rcptt.ini
/Users/Ulik_MacAir/Downloads/q7runner-1.5.0-SNAPSHOT/plugins/Users/Ulik_MacAir/Desktop/results/aut-rcptt-0_restarted_1: AUT arguments: -os ${target.os} -arch ${target.arch} -nl ${target.nl} -consoleLog
/Users/Ulik_MacAir/Downloads/q7runner-1.5.0-SNAPSHOT/plugins/Users/Ulik_MacAir/Desktop/results/aut-rcptt-0_restarted_1: AUT VM arguments: -Xms128m -Xmx768m -XX:PermSize=128M -XX:MaxPermSize=192M -XstartOnFirstThread -Dorg.eclipse.swt.internal.carbon.smallFonts
Fail 3 (5) processed. 1 failed. spent: 0:32, 0:06 mins remaining. 2_Test Case which executes AUT shutdown. time: 2093ms  Cause: Couldn't open ECL session
Fail 4 (5) processed. 2 failed. spent: 0:52, 0:00 mins remaining. 3_Failing Test Case. time: 3168ms  Cause: The Window "NonExistingWindow" could not be found.
Pass 5 (5) processed. 2 failed. spent: 0:57, 0:00 mins remaining. 4_Passing Test Case 2. time: 4815ms 
Failed Tests:
3_Failing Test Case
2_Test Case which executes AUT shutdown
Process terminated. Shut down AUTs
Show
Ulyana Skorokhodova added a comment - 29/Aug/14 4:21 PM Verified. Made a corresponding test suite (added one additional test case with File/Restart). The Runner's behaved as expected. Here is a full log:
Pass 1 (5) processed. 0 failed. spent: 0:06, 0:06 mins remaining. 1_Passing Test Case. time: 5269ms 
Pass 2 (5) processed. 0 failed. spent: 0:29, 0:14 mins remaining. 22_Test Case with File-Restart. time: 22719ms 
AUT-0:Restarting (restartId=1)
AUT-0:Product: org.eclipse.rcptt.platform.product
AUT-0:Application: org.eclipse.ui.ide.workbench
AUT-0:Architecture: x86_64
64bit arch is selected because AUT uses launcher library
	"../../../plugins/org.eclipse.equinox.launcher.cocoa.macosx.x86_64_1.1.101.v20120109-1504" specified in config file: rcptt.app/Contents/MacOS/rcptt.ini
/Users/Ulik_MacAir/Downloads/q7runner-1.5.0-SNAPSHOT/plugins/Users/Ulik_MacAir/Desktop/results/aut-rcptt-0_restarted_1: AUT arguments: -os ${target.os} -arch ${target.arch} -nl ${target.nl} -consoleLog
/Users/Ulik_MacAir/Downloads/q7runner-1.5.0-SNAPSHOT/plugins/Users/Ulik_MacAir/Desktop/results/aut-rcptt-0_restarted_1: AUT VM arguments: -Xms128m -Xmx768m -XX:PermSize=128M -XX:MaxPermSize=192M -XstartOnFirstThread -Dorg.eclipse.swt.internal.carbon.smallFonts
Fail 3 (5) processed. 1 failed. spent: 0:32, 0:06 mins remaining. 2_Test Case which executes AUT shutdown. time: 2093ms  Cause: Couldn't open ECL session
Fail 4 (5) processed. 2 failed. spent: 0:52, 0:00 mins remaining. 3_Failing Test Case. time: 3168ms  Cause: The Window "NonExistingWindow" could not be found.
Pass 5 (5) processed. 2 failed. spent: 0:57, 0:00 mins remaining. 4_Passing Test Case 2. time: 4815ms 
Failed Tests:
3_Failing Test Case
2_Test Case which executes AUT shutdown
Process terminated. Shut down AUTs
Ulyana Skorokhodova made changes - 29/Aug/14 4:21 PM
Resolution Fixed [ 1 ]
Status Open [ 1 ] Resolved [ 5 ]
Transition Time In Source Status Execution Times Last Executer Last Execution Date
Open Open Resolved Resolved
1d 19h 19m 1 Ulyana Skorokhodova 29/Aug/14 4:21 PM

People

Vote (0)
Watch (0)

Dates

  • Created:
    27/Aug/14 9:01 PM
    Updated:
    29/Aug/14 4:21 PM
    Resolved:
    29/Aug/14 4:21 PM