Jenkins running Python website functional tests

[docbop]

Trying to use Jenkins to run python-selenium-testng scripts to do
website functional testing and have Jenkins collect the testng
result.   (FYI this is on a ubuntu system)

Having problem getting Jenkins to run the python scripts with either
the build shell or the shiningPanda python plugin.    The errors from
the console are one or two depending on how I configure the tests.

1, the python source can't be found.   That appears to semi-solved
using the Custom Workplace directory.

2. The biggest issue is when Jenkins does find the source and run the
script I get errors that the selenium library can't be found by
python.

From testing the python script can be run from the command line by any
account except the Jenkins created account.   Something about the
shell Jenkins spawns to run tests in doesn't include the python lib's
everyone else can find.  I even su'd into the jenkins account and it
won't run the script like all the other accounts can.   ShiningPanda
python plugin has an option to use global lib's and checking that does
not help.

So could questions.....
1. Can Jenkins be used like this to run python scripts that launch
firefox and testng return results.

2. Assuming answer to question one is yes,  What do I need to do to
get the jenkins account to find and use the same lib's the rest of the
system can?

Thanks for any pointer or advise.

[danny staple]

Jenkins has its own environment setup - so you will need to add any custom pythonpath tweaks to the global environment on the Manage Jenkins -> Global Configuration page, or add it to a relevant slave node if only that node runs it.

On 17 February 2012 19:02, SteveB <[hidden email]> wrote:

Trying to use Jenkins to run python-selenium-testng scripts to do
website functional testing and have Jenkins collect the testng
result.   (FYI this is on a ubuntu system)

Having problem getting Jenkins to run the python scripts with either
the build shell or the shiningPanda python plugin.    The errors from
the console are one or two depending on how I configure the tests.

1, the python source can't be found.   That appears to semi-solved
using the Custom Workplace directory.

2. The biggest issue is when Jenkins does find the source and run the
script I get errors that the selenium library can't be found by
python.

From testing the python script can be run from the command line by any
account except the Jenkins created account.   Something about the
shell Jenkins spawns to run tests in doesn't include the python lib's
everyone else can find.  I even su'd into the jenkins account and it
won't run the script like all the other accounts can.   ShiningPanda
python plugin has an option to use global lib's and checking that does
not help.

So could questions.....
1. Can Jenkins be used like this to run python scripts that launch
firefox and testng return results.

2. Assuming answer to question one is yes,  What do I need to do to
get the jenkins account to find and use the same lib's the rest of the
system can?

Thanks for any pointer or advise.

--
Danny Staple

Director, ODM Solutions Ltd
w: http://www.odmsolutions.co.uk
Blog: http://orionrobots.co.uk/blog1-Danny-Staple

[Chris Withers]

On 17/02/2012 19:02, SteveB wrote:
> Trying to use Jenkins to run python-selenium-testng scripts to do
> website functional testing and have Jenkins collect the testng
> result.   (FYI this is on a ubuntu system)

Can you please include full examples of the tracebacks you're seeing?

(It'd also be good to know how you've configure python, how you've
installed the selenium python library you refer to, etc)

cheers,

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
            - http://www.simplistix.co.uk

[docbop]

It's a  holiday here and all the files are at work.   I can post info
tomorrow.

As to the selenium lib I used the pip install -U selenium like the
python/selenium doc page says.

Steve B.

On Feb 19, 5:06 am, Chris Withers <[hidden email]> wrote:

> On 17/02/2012 19:02, SteveB wrote:
>
> > Trying to useJenkinsto runpython-selenium-testng scripts to do
> > website functional testing and haveJenkinscollect the testng
> > result.   (FYI this is on a ubuntu system)
>
> Can you please include full examples of the tracebacks you're seeing?
>
> (It'd also be good to know how you've configurepython, how you've
> installed the seleniumpythonlibrary you refer to, etc)
>
> cheers,
>
> Chris
>
> --
> Simplistix - Content Management, Batch Processing &PythonConsulting
>             -http://www.simplistix.co.uk

[Steve Barnette]

Here's the config's and error messages I get when testing.

FYI: The python source is the selenium's example from thier doc.

In first two cases appears source code is found, but the selenium
isn't.
Selenium is installed and from command line any account can run the
program,
only except is under the Jenkins account. I can't re-install selenium
under
Jenkins because it is passworded.

Errors below were from build setup
Advanced Project Options
        Use custom workspace - /home/steveb/QA-project/Python
 This was done so Jenkins could find the source since not using any at
 this time.

Build  Virtual Builder
        Python version - System
        Nature - Shell
        Command - /usr/bin/python2.7 selePy2.py
        Advanced - System site packages checked

Error

Started by user anonymous
Building in workspace /home/steveb/QA-project/Python
[Python] $ /bin/bash -xe /tmp/shiningpanda602526163731452384.sh
+ /usr/bin/python2.7 selePy2.py
Traceback (most recent call last):
  File "selePy2.py", line 6, in <module>
    browser = webdriver.Firefox() # Get local session of firefox
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/
firefox/webdriver.py", line 45, in __init__
    self.binary, timeout),
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/
firefox/extension_connection.py", line 46, in __init__
    self.binary.launch_browser(self.profile)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/
firefox/firefox_binary.py", line 44, in launch_browser
    self._wait_until_connectable()
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/
firefox/firefox_binary.py", line 81, in _wait_until_connectable
    self._get_firefox_output())
selenium.common.exceptions.WebDriverException: Message: 'The browser
appears to have exited before we could connect. The output was: Error:
no display specified\n'
Build step 'Virtualenv Builder' marked build as failure
Finished: FAILURE

Switched to XShell and same errors as above.

Switched to Python and removed path to python, just the source
filename.

In  this case it can't seem to find the source file.

Started by user anonymous
Building in workspace /home/steveb/QA-project/Python
[Python] $ /var/lib/jenkins/shiningpanda/jobs/86790f80/virtualenvs/
d41d8cd9/bin/python /tmp/shiningpanda3316563476659644939.py
Traceback (most recent call last):
  File "/tmp/shiningpanda3316563476659644939.py", line 1, in <module>
    selePy2.py
NameError: name 'selePy2' is not defined
Build step 'Virtualenv Builder' marked build as failure
Finished: FAILURE



========================================================================
Switched the build step  to Python Builder

I put in full path to python and source file.

Same error of can't find selenium libs.


Only other error that sometimes pops up in trying all the various
Python
build variation is: "invalid interpter" and that appear to be a path
issue.







On Feb 20, 8:23 am, SteveB <[hidden email]> wrote:

> It's a  holiday here and all the files are at work.   I can post info
> tomorrow.
>
> As to the selenium lib I used the pip install -U selenium like thepython/selenium doc page says.
>
> Steve B.
>
> On Feb 19, 5:06 am, Chris Withers <[hidden email]> wrote:
>
>
>
>
>
>
>
> > On 17/02/2012 19:02, SteveB wrote:
>
> > > Trying to useJenkinsto runpython-selenium-testng scripts to do
> > > website functional testing and haveJenkinscollect the testng
> > > result.   (FYI this is on a ubuntu system)
>
> > Can you please include full examples of the tracebacks you're seeing?
>
> > (It'd also be good to know how you've configurepython, how you've
> > installed the seleniumpythonlibrary you refer to, etc)
>
> > cheers,
>
> > Chris
>
> > --
> > Simplistix - Content Management, Batch Processing &PythonConsulting
> >             -http://www.simplistix.co.uk

[Jminne-3]

can you run it from the command line on the slave?  I usually do that
to troubleshoot config issues like this.

On Feb 21, 3:00 pm, Steve Barnette <[hidden email]>
wrote:

> Here's the config's and error messages I get when testing.
>
> FYI: The python source is the selenium's example from thier doc.
>
> In first two cases appears source code is found, but the selenium
> isn't.
> Selenium is installed and from command line any account can run the
> program,
> only except is under the Jenkins account. I can't re-install selenium
> under
> Jenkins because it is passworded.
>
> Errors below were from build setup
> Advanced Project Options
>         Use custom workspace - /home/steveb/QA-project/Python
>  This was done so Jenkins could find the source since not using any at
>  this time.
>
> Build  Virtual Builder
>         Python version - System
>         Nature - Shell
>         Command - /usr/bin/python2.7 selePy2.py
>         Advanced - System site packages checked
>
> Error
>
> Started by user anonymous
> Building in workspace /home/steveb/QA-project/Python
> [Python] $ /bin/bash -xe /tmp/shiningpanda602526163731452384.sh
> + /usr/bin/python2.7 selePy2.py
> Traceback (most recent call last):
>   File "selePy2.py", line 6, in <module>
>     browser = webdriver.Firefox() # Get local session of firefox
>   File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/
> firefox/webdriver.py", line 45, in __init__
>     self.binary, timeout),
>   File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/
> firefox/extension_connection.py", line 46, in __init__
>     self.binary.launch_browser(self.profile)
>   File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/
> firefox/firefox_binary.py", line 44, in launch_browser
>     self._wait_until_connectable()
>   File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/
> firefox/firefox_binary.py", line 81, in _wait_until_connectable
>     self._get_firefox_output())
> selenium.common.exceptions.WebDriverException: Message: 'The browser
> appears to have exited before we could connect. The output was: Error:
> no display specified\n'
> Build step 'Virtualenv Builder' marked build as failure
> Finished: FAILURE
>
> Switched to XShell and same errors as above.
>
> Switched to Python and removed path to python, just the source
> filename.
>
> In  this case it can't seem to find the source file.
>
> Started by user anonymous
> Building in workspace /home/steveb/QA-project/Python
> [Python] $ /var/lib/jenkins/shiningpanda/jobs/86790f80/virtualenvs/
> d41d8cd9/bin/python /tmp/shiningpanda3316563476659644939.py
> Traceback (most recent call last):
>   File "/tmp/shiningpanda3316563476659644939.py", line 1, in <module>
>     selePy2.py
> NameError: name 'selePy2' is not defined
> Build step 'Virtualenv Builder' marked build as failure
> Finished: FAILURE
>
> ========================================================================
> Switched the build step  to Python Builder
>
> I put in full path to python and source file.
>
> Same error of can't find selenium libs.
>
> Only other error that sometimes pops up in trying all the various
> Python
> build variation is: "invalid interpter" and that appear to be a path
> issue.
>
> On Feb 20, 8:23 am, SteveB <[hidden email]> wrote:
>
>
>
>
>
>
>
> > It's a  holiday here and all the files are at work.   I can post info
> > tomorrow.
>
> > As to the selenium lib I used the pip install -U selenium like thepython/selenium doc page says.
>
> > Steve B.
>
> > On Feb 19, 5:06 am, Chris Withers <[hidden email]> wrote:
>
> > > On 17/02/2012 19:02, SteveB wrote:
>
> > > > Trying to useJenkinsto runpython-selenium-testng scripts to do
> > > > website functional testing and haveJenkinscollect the testng
> > > > result.   (FYI this is on a ubuntu system)
>
> > > Can you please include full examples of the tracebacks you're seeing?
>
> > > (It'd also be good to know how you've configurepython, how you've
> > > installed the seleniumpythonlibrary you refer to, etc)
>
> > > cheers,
>
> > > Chris
>
> > > --
> > > Simplistix - Content Management, Batch Processing &PythonConsulting
> > >             -http://www.simplistix.co.uk

[Chris Withers]

Hi Steve,

On 21/02/2012 21:00, Steve Barnette wrote:
> Build  Virtual Builder
> Python version - System
> Nature - Shell
> Command - /usr/bin/python2.7 selePy2.py

Don't specify the python path like this.
Use $PYTHON_EXE selePy2.py instead.

That said, what do the contents of selePy2.py look like?

How did you install the selenium python package?


>    File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/
> firefox/firefox_binary.py", line 81, in _wait_until_connectable
>      self._get_firefox_output())
> selenium.common.exceptions.WebDriverException: Message: 'The browser
> appears to have exited before we could connect. The output was: Error:
> no display specified\n'

This seems pretty clear. I don't know how you specify a display to
Selenium, but you need to.

> Switched to Python and removed path to python, just the source
> filename.

Still wrong, see above...
Reading the docs wouldn't be such a terrible idea you know ;-)

> Started by user anonymous
> Building in workspace /home/steveb/QA-project/Python
> [Python] $ /var/lib/jenkins/shiningpanda/jobs/86790f80/virtualenvs/
> d41d8cd9/bin/python /tmp/shiningpanda3316563476659644939.py
> Traceback (most recent call last):
>    File "/tmp/shiningpanda3316563476659644939.py", line 1, in<module>
>      selePy2.py
> NameError: name 'selePy2' is not defined

Yeah, when you select "python" it doesn't run a script, the contents you
type in are executed as python.

> ========================================================================
> Switched the build step  to Python Builder
>
> I put in full path to python and source file.

Still wrong, as indicated by the docs.

> Only other error that sometimes pops up in trying all the various
> Python
> build variation is: "invalid interpter" and that appear to be a path
> issue.

I doubt that ;-)

cheers,

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
             - http://www.simplistix.co.uk