urlencoded Forward slash is breaking URL

About the system

I have URLs of this format in my project:-

http://project_name/browse_by_exam/type/tutor_search/keyword/class/new_search/1/search_exam/0/search_subject/0

Where keyword/class pair means search with "class" keyword.

I have a common index.php file which executes for every module in the project. There is only a rewrite rule to remove the index.php from URL:-

RewriteCond $1 !^(index\.php|resources|robots\.txt)
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php [L,QSA]

I am using urlencode() while preparing the search URL and urldecode() while reading the search URL.

Problem

Only the forward slash character is breaking URLs causing 404 page not found error. For example, if I search one/two the URL is

http://project_name/browse_by_exam/type/tutor_search/keyword/one%2Ftwo/new_search/1/search_exam/0/search_subject/0/page_sort/

How do I fix this? I need to keep index.php hidden in the URL. Otherwise, if that was not needed, there would have been no problem with forward slash and I could have used this URL:-

http://project_name/index.php?browse_by_exam/type/tutor_search/keyword/one
%2Ftwo/new_search/1/search_exam/0/search_subject/0

Apache denies all URLs with %2F in the path part, for security reasons: scripts can't normally (ie. without rewriting) tell the difference between %2F and / due to the PATH_INFO environment variable being automatically URL-decoded (which is stupid, but a long-standing part of the CGI specification so there's nothing can be done about it).

You can turn this feature off using the AllowEncodedSlashes directive, but note that other web servers will still disallow it (with no option to turn that off), and that other characters may also be taboo (eg. %5C), and that %00 in particular will always be blocked by both Apache and IIS. So if your application relied on being able to have %2F or other characters in a path part you'd be limiting your compatibility/deployment options.

> I am using urlencode() while preparing the search URL

You should use rawurlencode(), not urlencode() for escaping path parts. urlencode() is misnamed, it is actually for application/x-www-form-urlencoded data such as in the query string or the body of a POST request, and not for other parts of the URL.

The difference is that + doesn't mean space in path parts. rawurlencode() will correctly produce %20 instead, which will work both in form-encoded data and other parts of the URL.

Replace %2F with %252F after url encoding

PHP

function custom_http_build_query($query=array()){

	return str_replace('%2F','%252F', http_build_query($query));
}

Handle the request via htaccess

.htaccess

RewriteCond %{REQUEST_URI} ^(.*?)(%252F)(.*?)$ [NC]
RewriteRule . %1/%3 [R=301,L,NE]

Resources

http://www.leakon.com/archives/865

In Apache, AllowEncodedSlashes On would prevent the request from being immediately rejected with a 404.

Just another idea on how to fix this.

$encoded_url = str_replace('%2F', '/', urlencode($url));

I had the same problem with slash in url get param, in my case following php code works:

$value = "hello/world"
$value = str_replace('/', '/', $value;?>
$value = urlencode($value);?>
# $value is now hello%26%2347%3Bworld

I first replace the slash by html entity and then I do the url encoding.

Here's my humble opinion. !!!! Don't !!!! change settings on the server to make your parameters work correctly. This is a time bomb waiting to happen someday when you change servers.

The best way I have found is to just convert the parameter to base 64 encoding. So in my case, I'm calling a php service from Angular and passing a parameter that could contain any value.

So my typescript code in the client looks like this:

    private encodeParameter(parm:string){
    if (!parm){
        return null;
    }
    return btoa(parm);
}

And to retrieve the parameter in php:

    $item_name = $request->getAttribute('item_name');
    $item_name = base64_decode($item_name); 

On my hosting account this problem was caused by a ModSecurity rule that was set for all accounts automatically. Upon my reporting this problem, their admin quickly removed this rule for my account.

Use a different character and replace the slashes server side

e.g. Drupal.org uses %21 (the excalamation mark character !) to represent the slash in a url parameter.

Both of the links below work:

https://api.drupal.org/api/drupal/includes%21common.inc/7

https://api.drupal.org/api/drupal/includes!common.inc/7

If you're worried that the character may clash with a character in the parameter then use a combination of characters.

So your url would be http://project_name/browse_by_exam/type/tutor_search/keyword/one_-!two/new_search/1/search_exam/0/search_subject/0

change it out with js and convert it back to a slash server side.

is simple for me use base64_encode

$term = base64_encode($term) 
$url = $youurl.'?term='.$term

after you decode the term

$term = base64_decode($['GET']['term'])

this way encode the "/" and ""

A standard solution for this problem is to allow slashes by making the parameter that may contain slashes the last parameter in the url.

For a product code url you would then have...

mysite.com/product/details/PR12345/22

For a search term you'd have

http://project/search_exam/0/search_subject/0/keyword/Psychology/Management

(The keyword here is Psychology/Management)

It's not a massive amount of work to process the first "named" parameters then concat the remaining ones to be product code or keyword.

Some frameworks have this facility built in to their routing definitions.

This is not applicable to use case involving two parameters that my contain slashes.

I use javascript encodeURI() function for the URL part that has forward slashes that should be seen as characters instead of http address. Eg:

"/api/activites/" + encodeURI("?categorie=assemblage&nom=Manipulation/Finition")

see http://www.w3schools.com/tags/ref_urlencode.asp

I solved this by using 2 custom functions like so:

function slash_replace($query){

    return str_replace('/','_', $query);
}

function slash_unreplace($query){

    return str_replace('_','/', $query);
}

So to encode I could call:

rawurlencode(slash_replace($param))

and to decode I could call

slash_unreplace(rawurldecode($param);

Cheers!

You can use %2F if using it this way:
?param1=value1&param2=value%2Fvalue

but if you use /param1=value1/param2=value%2Fvalue it will throw an error.