How to uninstall Homebrew Cask

I installed Homebrew Cask for kicks as described at http://caskroom.io/. It boils down to the following command:

$ brew install caskroom/cask/brew-cask
$ brew tap caskroom/versions

Homebrew Cask installed and everything was fine. Then I uninstalled Homebrew Cask like so:

$ brew untap caskroom/versions
$ brew uninstall brew-cask

Here are the current taps:

$ brew tap
homebrew/versions

Yet I see stuff from Cask when searching for items that wouldn't be part of Homebrew itself (like Graphical programs, aText, Fantastical etc.):

$ brew search atext
Caskroom/cask/atext

Though it doesn't install (as expected):

$ brew install atext 
Error: No available formula for atext
Searching formulae...
Searching taps...
Caskroom/cask/atext

So the question is why does brew search Caskroom/cask and how do I get rid of it (I don't want Homebrew telling me it has aText, for example, when it can't install it)?

I am running Mac OS X Yosemite.

For the current version of Brew:

$ brew uninstall --cask yed

For older versions of Brew:

$ brew cask uninstall yed

the result:

==> Removing App symlink: '/Users/user/Applications/yEd.app'

As of Homebrew version 0.9.5, it appears that brew search will find items in Caskroom/cask/ without actually having Homebrew Cask installed. You can also install them, but this will actually install brew-cask as a dependency along the way.

Try:

brew install Caskroom/cask/atext

Command the terminal to "untap" at the prompt [MacOS Catalina 10.15.2]

Run

brew untap homebrew/cask-fonts

Expected output

Untapping homebrew/cask-fonts...
Untapped 1571 casks (1,713 files, 11.4MB)