How can I escape an exclamation mark ! in cmd scripts?

When I have setlocal ENABLEDELAYEDEXPANSION set in a cmd script is there any way I can escape a ! that I want to use as a parameter to a command?

@echo off
setlocal ENABLEDELAYEDEXPANSION
echo I want to go out with a bang!
echo I still want to go out with a bang^!

That's what I found (^^)

@echo off
setlocal ENABLEDELAYEDEXPANSION
echo I want to go out with a bang^^!

An additional remark to the answer of FrVaBe.

Normally the ^^! works, but in quotes you only need ^! instead.

echo I want to go out with a bang^^!
echo He said "Bang^!"

This is a result of the escape mechanism of the batch parser.

First the parser parses a line and the caret escapes the next character, in this case it has an effect for &|<>()"<linefeed>, but only outside of quotes, as inside of the quotes all characters are "normal" and the caret itself has no effect.

With delayed expansion an extra parse step follows, there is the caret also an escape character for the next character, but only affects the ! and ^, and quotes are ignored in this parsing step. This extra step will be executed only, if there is at least one ! in the line.

setlocal DisableDelayedExpansion
echo DisableDelayedExpansion
echo one caret^^
echo one caret^^  bang! "boom^!"

echo(
setlocal EnableDelayedExpansion
echo EnableDelayedExpansion
echo one caret^^
echo none caret^^  bang^^! "boom^!"

---- OUTPUT ------

DisableDelayedExpansion
one caret^
one caret^  bang! "boom^!"

EnableDelayedExpansion
one caret^
none caret  bang! "boom!"

EDIT

Here is a slightly modified example that better illustrates the various escape permutations that are required, depending on the context. The only case that requires unusual escaping is the last example when delayed expansion is on and there exists at least one ! on the line.

@echo off
setlocal DisableDelayedExpansion
echo DisableDelayedExpansion
echo caret^^       "caret^"
echo caret^^ bang! "caret^ bang!"

echo(
setlocal EnableDelayedExpansion
echo EnableDelayedExpansion
echo caret^^       "caret^"
echo caret^^^^ bang^^! "caret^^ bang^!"

-- OUTPUT --

DisableDelayedExpansion
caret^       "caret^"
caret^ bang! "caret^ bang!"

EnableDelayedExpansion
caret^       "caret^"
caret^ bang! "caret^ bang!"

To use an explanation point in batch with Delayed Expansion enabled, you must first add the explanation point to a variable with it disabled. See the below example with both DISABLEDELAYEDEXPANSION and ENABLEDELAYEDEXPANSION state.

@echo off
setlocal DISABLEDELAYEDEXPANSION
set DB_password=encrypt!Pws
echo %DB_password%
SETLOCAL ENABLEDELAYEDEXPANSION
echo !DB_password!

Thanks. To add to this valuable point, if one's script contains a variable whose value contains an "!", then the following approach will render that value as-is:

@echo off
SETLOCAL EnableDelayedExpansion
set /P omg=Enter a value that contains an exclamation-point:
echo Traditional: %omg%
echo Alternative: !omg!
pause