Batch character escaping

I'm fairly proficient at writing Batch scripts for Windows, but even after all these years how to correctly escape characters puzzles me. It's especially difficult when trying figure out the correct way to escape a regular expression for use with sed. Is there any tool that can help me? Perhaps something that allows me to paste in a "normal" string and it spits out the correctly escaped version of that string?

Update: I'm reluctant to give an example because I'm not looking for an answer on how to escape one specific string. I'm also not looking for a solution that will work for one specific app. I'm looking for a tool that will help me get the escape syntax correct for every string I ever need to escape no matter what tool might be consuming it from the command line.

That being said the regex I really want is

(^.*)(Form Product=")([^"]*") FormType="[^"]*" FormID="([0-9][0-9]*)".*$

Take that true regex (i.e. unescaped as far as BATCH is concerned) and wrap it in some sed syntax such as ssed "s@ --- Insert escaped regex here --- @http://psph/\1/\2@g" "%~1" and finally escape it... Again, is there any tool that can assist in escaping any string for use on the BATCH command line?

p.s. There are so many exceptions to BATCH's escaping syntax that I'll even settle for a good cheat sheet.

As dbhenham points out in this comment, a (MUCH) more detailed answer can be found in portions of this answer (originally by another user jeb and significantly edited and updated by dbhenham since) on a related but much more general question:

• parsing - How does the Windows Command Interpreter (CMD.EXE) parse scripts? - Stack Overflow

Note that, per dbhenham, this answer is:

> incorrect, misleading, and incomplete

I think this answer is still good enough, for almost all cases, but a careful reading of the above answer might be warranted depending on one's exact character escaping needs and the limitations of this answer.

The remaining has been adapted with permission of the author from the page Batch files - Escape Characters on Rob van der Woude's Scripting Pages site.

TLDR

Windows (and DOS) batch file character escaping is complicated:

> > Much like the universe, if anyone ever does fully come to understand Batch then the language will instantly be replaced by an infinitely weirder and more complex version of itself. This has obviously happened at least once before ;)

Percent Sign %

% can be escaped as %% – "May not always be required [to be escaped] in doublequoted strings, just try"

Generally, Use a Caret ^

These characters "may not always be required [to be escaped] in doublequoted strings, but it won't hurt":

• ^
• &
• <
• >
• |

Example: echo a ^> b to print a > b on screen

' is "required [to be escaped] only in the FOR /F "subject" (i.e. between the parenthesis), unless backq is used"

` is "required [to be escaped] only in the FOR /F "subject" (i.e. between the parenthesis), if backq is used"

These characters are "required [to be escaped] only in the FOR /F "subject" (i.e. between the parenthesis), even in doublequoted strings":

• ,
• ;
• =
• (
• )

Double Escape Exclamation Points when Using Delayed Variable Expansion

! must be escaped ^^! when delayed variable expansion is active.

Double Double-Quotes in find Search Patterns

" → ""

Use a Backslash in findstr Regex Patterns

• \
• [
• ]
• "
• .
• *
• ?

Also

Rob commented further on this question (via email correspondence with myself):

> As for the answer, I'm afraid the chaos is even worse than the original poster realizes: requirements for escaping parentheses also depend on the string being inside a code block or not! > > I guess an automated tool could just insert a caret before every character, then doubling all percent signs - and it would still fail if the string is doublequoted!

Further, individual programs are responsible for parsing their command line arguments so some of the escaping required for, e.g. for sed or ssed, may be due to the specific programs called in the batch scripts.

The escape character for batch is the caret (^). If you want to include any of the pipeline characters in your script you need to prefix the character with the caret:

:: Won't work:
@echo Syntax: MyCommand > [file]

:: Will work:
@echo Syntax: MyCommand ^> [file]

You could simply use an external file as input for sed.

Or using strings directly in batch, it's a good idea to use the delayed expansion.

setlocal DisableDelayedExpansion
set "regEx=s/^#*$/""/g"
setlocal EnableDelayedExpansion
sed !regEx! file.txt

EDIT: How to use unmodified strings with a batch

This uses findstr to get the string directly from the batch and return it into a result-variable.
So you can use the sed-string as is.

@echo off
setlocal
REM SedString1#(^.*)(Form Product=")([^"]*") FormType="[^"]*" FormID="([0-9][0-9]*)".*$

call :GetSEDString result SedString1
setLocal EnableDelayedExpansion
echo the sedString is !result!
sed !result!
goto :eof

:GetSEDString <resultVar> <searchName>
:: Search the own batch file for <searchName> in a line with "REM <searchName>#"
:: Return all after the "#" without any modification
setLocal DisableDelayedExpansion
for /f "usebackq tokens=* delims=" %%G in (`findstr /n /c:"REM %~2#" "%~f0"`) do (
	set "str=%%G"
)
setLocal EnableDelayedExpansion
set "str=!str:*#=!"

for /F "delims=" %%A in ("!str!") DO (
  endlocal
  endlocal
  set "%~1=%%A"
  goto :eof
)

goto :eof

A simple solution to preserve all command line arguments is to use %*: it returns the whole command line starting at the first command line argument (in Windows NT 4, %* also includes all leading spaces) and excluding any output redirection.