Is there any way to start with a POST request using Selenium?
SeleniumSelenium Problem Overview
I'm trying to start a Selenium test with a POST request to my application.
Instead of a simple open(/startpoint)
I would like to do something like open(/startpoint, stuff=foo,stuff2=bar)
Is there any way to do that?
I'm asking this because the original page which posts to this start point depends on external providers that are often offline (development environment) and so will often fail too early (and are not the subject of the test)
I guess sending data as GET would work too. I would just prefer using a POST method.
Selenium Solutions
Solution 1 - Selenium
If you are using Python selenium bindings, nowadays, there is an extension to selenium - selenium-requests:
> Extends Selenium WebDriver classes to include the request function > from the Requests library, while doing all the needed cookie and > request headers handling.
Example:
from seleniumrequests import Firefox
webdriver = Firefox()
response = webdriver.request('POST', 'url here', data={"param1": "value1"})
print(response)
Solution 2 - Selenium
Short answer: No.
But you might be able to do it with a bit of filthing. If you open up a test page (with GET) then evaluate some JavaScript on that page you should be able to replicate a POST request. See https://stackoverflow.com/q/133925/569659 to see how you can replicate a POST request in JavaScript.
Hope this helps.
Solution 3 - Selenium
One very practical way to do this is to create a dummy start page for your tests that is simply a form with POST that has a single "start test" button and a bunch of <input type="hidden"... elements with the appropriate post data.
For example you might create a SeleniumTestStart.html page with these contents:
<body>
<form action="/index.php" method="post">
<input id="starttestbutton" type="submit" value="starttest"/>
<input type="hidden" name="stageid" value="stage-you-need-your-test-to-start-at"/>
</form>
</body>
In this example, index.php is where your normal web app is located.
The Selenium code at the start of your tests would then include:
open /SeleniumTestStart.html
clickAndWait starttestbutton
This is very similar to other mock and stub techniques used in automated testing. You are just mocking the entry point to the web app.
Obviously there are some limitations to this approach:
- data cannot be too large (e.g. image data)
- security might be an issue so you need to make sure that these test files don't end up on your production server
- you may need to make your entry points with something like php instead of html if you need to set cookies before the Selenium test gets going
- some web apps check the referrer to make sure someone isn't hacking the app - in this case this approach probably won't work - you may be able to loosen this checking in a dev environment so it allows referrers from trusted hosts (not self, but the actual test host)
Please consider reading my article about the Qualities of an Ideal Test
Solution 4 - Selenium
Selenium IDE allows you to run Javascript using storeEval command. Mentioned above solution works fine if you have test page (HTML, not XML) and you need to perform only POST request.
If you need to make POST/PUT/DELETE or any other request then you will need another approach:
XMLHttpRequest!
Example listed below has been tested - all methods (POST/PUT/DELETE) work just fine.
<!--variables-->
<tr>
<td>store</td>
<td>/your/target/script.php</td>
<td>targetUrl</td>
</tr>
<tr>
<td>store</td>
<td>user=user1&password</td>
<td>requestParams</td>
</tr>
<tr>
<td>store</td>
<td>POST</td>
<td>requestMethod</td>
</tr>
<!--scenario-->
<tr>
<td>storeEval</td>
<td>window.location.host</td>
<td>host</td>
</tr>
<tr>
<td>store</td>
<td>http://${host}</td>
<td>baseUrl</td>
</tr>
<tr>
<td>store</td>
<td>${baseUrl}${targetUrl}</td>
<td>absoluteUrl</td>
</tr>
<tr>
<td>store</td>
<td>${absoluteUrl}?${requestParams}</td>
<td>requestUrl</td>
</tr>
<tr>
<td>storeEval</td>
<td>var method=storedVars['requestMethod']; var url = storedVars['requestUrl']; loadXMLDoc(url, method); function loadXMLDoc(url, method) { var xmlhttp = new XMLHttpRequest(); xmlhttp.onreadystatechange=function() { if (xmlhttp.readyState==4) { if(xmlhttp.status==200) { alert("Results = " + xmlhttp.responseText);} else { alert("Error!"+ xmlhttp.responseText); }}}; xmlhttp.open(method,url,true); xmlhttp.send(); }</td>
<td></td>
</tr>
Clarification:
${requestParams} - parameters you would like to post (e.g. param1=value1¶m2=value3¶m1=value3) you may specify as many parameters as you need
${targetUrl} - path to your script (if your have page located at http://domain.com/application/update.php then targetUrl should be equal to /application/update.php)
${requestMethod} - method type (in this particular case it should be "POST" but can be "PUT" or "DELETE" or any other)
Solution 5 - Selenium
Selenium doesn't currently offer API for this, but there are several ways to initiate an HTTP request in your test. It just depends what language you are writing in.
In Java for example, it might look like this:
// setup the request
String request = "startpoint?stuff1=foo&stuff2=bar";
URL url = new URL(request);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("POST");
// get a response - maybe "success" or "true", XML or JSON etc.
InputStream inputStream = connection.getInputStream();
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
String line;
StringBuffer response = new StringBuffer();
while ((line = bufferedReader.readLine()) != null) {
response.append(line);
response.append('\r');
}
bufferedReader.close();
// continue with test
if (response.toString().equals("expected response"){
// do selenium
}
Solution 6 - Selenium
Well, i agree with the @Mishkin Berteig - Agile Coach answer. Using the form is the quick way to use the POST features.
Anyway, i see some mention about javascript, but no code. I have that for my own needs, which includes jquery for easy POST plus others.
Basically, using the driver.execute_script() you can send any javascript, including Ajax queries.
#/usr/local/env/python
# -*- coding: utf8 -*-
# proxy is used to inspect data involved on the request without so much code.
# using a basic http written in python. u can found it there: http://voorloopnul.com/blog/a-python-proxy-in-less-than-100-lines-of-code/
import selenium
from selenium import webdriver
import requests
from selenium.webdriver.common.proxy import Proxy, ProxyType
jquery = open("jquery.min.js", "r").read()
#print jquery
proxy = Proxy()
proxy.proxy_type = ProxyType.MANUAL
proxy.http_proxy = "127.0.0.1:3128"
proxy.socks_proxy = "127.0.0.1:3128"
proxy.ssl_proxy = "127.0.0.1:3128"
capabilities = webdriver.DesiredCapabilities.PHANTOMJS
proxy.add_to_capabilities(capabilities)
driver = webdriver.PhantomJS(desired_capabilities=capabilities)
driver.get("http://httpbin.org")
driver.execute_script(jquery) # ensure we have jquery
ajax_query = '''
$.post( "post", {
"a" : "%s",
"b" : "%s"
});
''' % (1,2)
ajax_query = ajax_query.replace(" ", "").replace("\n", "")
print ajax_query
result = driver.execute_script("return " + ajax_query)
#print result
#print driver.page_source
driver.close()
# this retuns that from the proxy, and is OK
'''
POST http://httpbin.org/post HTTP/1.1
Accept: */*
Referer: http://httpbin.org/
Origin: http://httpbin.org
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/538.1 (KHTML, like Gecko) PhantomJS/2.0.0 Safari/538.1
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Content-Length: 7
Cookie: _ga=GAxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx; _gat=1
Connection: Keep-Alive
Accept-Encoding: gzip, deflate
Accept-Language: es-ES,en,*
Host: httpbin.org
None
a=1&b=2 <<---- that is OK, is the data contained into the POST
None
'''
Solution 7 - Selenium
I used driver.execute_script() to inject an html form into the page, and then submit it. It looks like this:
def post(path, params):
driver.execute_script("""
function post(path, params, method='post') {
const form = document.createElement('form');
form.method = method;
form.action = path;
for (const key in params) {
if (params.hasOwnProperty(key)) {
const hiddenField = document.createElement('input');
hiddenField.type = 'hidden';
hiddenField.name = key;
hiddenField.value = params[key];
form.appendChild(hiddenField);
}
}
document.body.appendChild(form);
form.submit();
}
post(arguments[1], arguments[0]);
""", params, path)
# example
post(path='/submit', params={'name': 'joe'})
If you'd like, you can just add it the function to \selenium\webdriver\chrome\webdriver.pyand then use it in your code with driver.post()
Solution 8 - Selenium
from selenium import webdriver
driver = webdriver.Firefox()
driver.implicitly_wait(12)
driver.set_page_load_timeout(10)
def _post_selenium(self, url: str, data: dict):
input_template = '{k} <input type="text" name="{k}" id="{k}" value="{v}"><BR>\n'
inputs = ""
if data:
for k, v in data.items():
inputs += input_template.format(k=k, v=v)
html = f'<html><body>\n<form action="{url}" method="post" id="formid">\n{inputs}<input type="submit" id="inputbox">\n</form></body></html>'
html_file = os.path.join(os.getcwd(), 'temp.html')
with open(html_file, "w") as text_file:
text_file.write(html)
driver.get(f"file://{html_file}")
driver.find_element_by_id('inputbox').click()
_post_selenium("post.to.my.site.url", {"field1": "val1"})
driver.close()