How to prevent Google Colab from disconnecting?
PythonLinuxJupyter NotebookGoogle ColaboratoryPython Problem Overview
Q: Is there any way to programmatically prevent Google Colab from disconnecting on a timeout?
The following describes the conditions causing a notebook to automatically disconnect:
> Google Colab notebooks have an idle timeout of 90 minutes and absolute timeout of 12 hours. This means, if user does not interact with his Google Colab notebook for more than 90 minutes, its instance is automatically terminated. Also, maximum lifetime of a Colab instance is 12 hours.
Naturally, we want to automatically squeeze the maximum out of the instance, without having to manually interact with it constantly. Here I will assume commonly seen system requirements:
- Ubuntu 18 LTS / Windows 10 / Mac Operating systems
- In case of Linux-based systems, using popular DEs like Gnome 3 or Unity
- Firefox or Chromium browsers
I should point out here that such behavior does not violate Google Colab's Terms of Use, although it is not encouraged according to their FAQ (in short: morally it is not okay to use up all of the GPUs if you don't really need it).
My current solution is very dumb:
- First, I turn the screensaver off, so my sreen is always on.
- I have an Arduino board, so I just turned it into a rubber ducky usb and make it emulate primitive user interaction while I sleep (just because I have it at hand for other use-cases).
Are there better ways?
Python Solutions
Solution 1 - Python
Edit 2: As of March 2021, none of these methods will work as google added a captcha button that randomly pops up after some time.
Edit 1: Apparently the solution is very easy, and doesn't need any JavaScript. Just create a new cell at the bottom having the following line:
while True:pass
now keep the cell in the run sequence so that the infinite loop won't stop and thus keep your session alive.
Old method: Set a javascript interval to click on the connect button every 60 seconds. Open developer-settings (in your web-browser) with Ctrl+Shift+I then click on console tab and type this on the console prompt. (for mac press Option+Command+I)
function ConnectButton(){
console.log("Connect pushed");
document.querySelector("#top-toolbar > colab-connectbutton").shadowRoot.querySelector("#connect").click()
}
setInterval(ConnectButton,60000);
Solution 2 - Python
Since the id of the connect button is now changed to "colab-connect-button", the following code can be used to keep clicking on the button.
function ClickConnect(){
console.log("Clicked on connect button");
document.querySelector("colab-connect-button").click()
}
setInterval(ClickConnect,60000)
If still, this doesn't work, then follow the steps below:
- Right-click on the connect button (on the top-right side of the colab)
- Click on inspect
- Get the HTML id of the button and substitute in the following code
function ClickConnect(){
console.log("Clicked on connect button");
document.querySelector("Put ID here").click() // Change id here
}
setInterval(ClickConnect,60000)
Solution 3 - Python
For me the following examples:
document.querySelector("#connect").click()
ordocument.querySelector("colab-toolbar-button#connect").click()
ordocument.querySelector("colab-connect-button").click()
were throwing errors.
I had to adapt them to the following:
Version 1:
function ClickConnect(){
console.log("Connnect Clicked - Start");
document.querySelector("#top-toolbar > colab-connect-button").shadowRoot.querySelector("#connect").click();
console.log("Connnect Clicked - End");
};
setInterval(ClickConnect, 60000)
Version 2: If you would like to be able to stop the function, here is the new code:
var startClickConnect = function startClickConnect(){
var clickConnect = function clickConnect(){
console.log("Connnect Clicked - Start");
document.querySelector("#top-toolbar > colab-connect-button").shadowRoot.querySelector("#connect").click();
console.log("Connnect Clicked - End");
};
var intervalId = setInterval(clickConnect, 60000);
var stopClickConnectHandler = function stopClickConnect() {
console.log("Connnect Clicked Stopped - Start");
clearInterval(intervalId);
console.log("Connnect Clicked Stopped - End");
};
return stopClickConnectHandler;
};
var stopClickConnect = startClickConnect();
In order to stop, call:
stopClickConnect();
Solution 4 - Python
Well this is working for me -
run the following code in the console and it will prevent you from disconnecting. Ctrl+ Shift + i to open inspector view . Then go to console.
function ClickConnect(){
console.log("Working");
document.querySelector("colab-toolbar-button#connect").click()
}
setInterval(ClickConnect,60000)
Solution 5 - Python
create a python code in your pc with pynput
from pynput.mouse import Button, Controller
import time
mouse = Controller()
while True:
mouse.click(Button.left, 1)
time.sleep(30)
Run this code in your Desktop, Then point mouse arrow over (colabs left panel - file section) directory structure on any directory this code will keep clicking on directory on every 30 seconds so it will expand and shrink every 30 seconds so your session will not get expired Important - you have to run this code in your pc
Solution 6 - Python
Instead of clicking the connect button, i just clicking on comment button to keep my session alive. (August-2020)
function ClickConnect(){
console.log("Working");
document.querySelector("#comments > span").click()
}
setInterval(ClickConnect,5000)
Solution 7 - Python
I use a Macro Program to periodically click on the RAM/Disk button to train the model all night. The trick is to configure a macro program to click on the Ram/Disk Colab Toolbar Button twice with a short interval between the two clicks so that even if the Runtime gets disconnected it will reconnect back. (the first click used to close the dialog box and the second click used to RECONNECT). However, you still have to leave your laptop open all night and maybe pin the Colab tab.
Solution 8 - Python
The above answers with the help of some scripts maybe work well. I have a solution(or a kind of trick) for that annoying disconnection without scripts, especially when your program must read data from your google drive, like training a deep learning network model, where using scripts to do reconnect
operation is of no use because once you disconnect with your colab, the program is just dead, you should manually connect to your google drive again to make your model able to read dataset again, but the scripts will not do that thing.
I've already test it many times and it works well.
When you run a program on the colab page with a browser(I use Chrome), just remember that don't do any operation to your browser once your program starts running, like: switch to other webpages, open or close another webpage, and so on, just just leave it alone there and waiting for your program finish running, you can switch to another software, like pycharm to keep writing your codes but not switch to another webpage. I don't know why open or close or switch to other pages will cause the connection problem of the google colab page, but each time I try to bothered my browser, like do some search job, my connection to colab will soon break down.
Solution 9 - Python
Try this:
function ClickConnect(){
console.log("Working");
document
.querySelector("#top-toolbar > colab-connect-button")
.shadowRoot
.querySelector("#connect")
.click()
}
setInterval(ClickConnect,60000)
Solution 10 - Python
Perhaps many of the previous solutions are no longer working. For example, this bellow code continues to create new code cells in Colab, working though. Undoubtedly, creating a bunch of code cells is an inconvenience. If too many code cells are created in some hours of running and there is no enough RAM, the browser may freeze.
This repetedly creates code cells—
function ClickConnect(){
console.log("Working");
document.querySelector("colab-toolbar-button").click()
}setInterval(ClickConnect,60000)
But I found the code below is working, it doesn't cause any problems. In the Colab notebook tab, click on the Ctrl + Shift + i
key simultaneously and paste the below code in the console. 120000 intervals are enough.
function ClickConnect(){
console.log("Working");
document.querySelector("colab-toolbar-button#connect").click()
}setInterval(ClickConnect,120000)
I have tested this code in firefox, in November 2020. It will work on chrome too.
Solution 11 - Python
I don't believe the JavaScript solutions work anymore. I was doing it from within my notebook with:
from IPython.display import display, HTML
js = ('<script>function ConnectButton(){ '
'console.log("Connect pushed"); '
'document.querySelector("#connect").click()} '
'setInterval(ConnectButton,3000);</script>')
display(HTML(js))
When you first do a Run all (before the JavaScript or Python code has started), the console displays:
Connected to
wss://colab.research.google.com/api/kernels/0e1ce105-0127-4758-90e48cf801ce01a3/channels?session_id=5d8...
However, ever time the JavaScript runs, you see the console.log portion, but the click portion simply gives:
Connect pushed
Uncaught TypeError: Cannot read property 'click' of null
at ConnectButton (<anonymous>:1:92)
Others suggested the button name has changed to #colab-connect-button, but that gives same error.
After the runtime is started, the button is changed to show RAM/DISK, and a drop down is presented. Clicking on the drop down creates a new <DIV class=goog menu...>
that was not shown in the DOM previously, with 2 options "Connect to hosted runtime" and "Connect to local runtime". If the console window is open and showing elements, you can see this DIV appear when you click the dropdown element. Simply moving the mouse focus between the two options in the new window that appears adds additional elements to the DOM, as soon as the mouse looses focus, they are removed from the DOM completely, even without clicking.
Solution 12 - Python
Using python selenium
from selenium.webdriver.common.keys import Keys
from selenium import webdriver
import time
driver = webdriver.Chrome('/usr/lib/chromium-browser/chromedriver')
notebook_url = ''
driver.get(notebook_url)
# run all cells
driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL + Keys.F9)
time.sleep(5)
# click to stay connected
start_time = time.time()
current_time = time.time()
max_time = 11*59*60 #12hours
while (current_time - start_time) < max_time:
webdriver.ActionChains(driver).send_keys(Keys.ESCAPE).perform()
driver.find_element_by_xpath('//*[@id="top-toolbar"]/colab-connect-button').click()
time.sleep(30)
current_time = time.time()
Solution 13 - Python
Updating (jul 21).
function ConnectButton(){
console.log("Working");
document.querySelector("#connect").click()
}
setInterval(ConnectButton,60000);
Solution 14 - Python
I tried the codes above but they did not work for me. So here is my JS code for reconnecting.
let interval = setInterval(function(){
let ok = document.getElementById('ok');
if(ok != null){
console.log("Connect pushed");
ok.click();
}},60000)
You can use it with the same way (run it on the console of your browser) to run it.
If you want to stop the script, you can enter clearInterval(interval)
and want to run again setInterval(interval)
.
I hope this helps you.
Solution 15 - Python
This one worked for me (it seems like they changed the button classname or id) :
function ClickConnect(){
console.log("Working");
document.querySelector("colab-connect-button").click()
}
setInterval(ClickConnect,60000)
Solution 16 - Python
Updated one. it works for me.
function ClickConnect(){
console.log("Working");
document.querySelector("paper-icon-button").click()
} Const myjob = setInterval(ClickConnect, 60000)
If isn't working you for you guys try clear it by running:
clearInterval(myjob)
Solution 17 - Python
you can bookmark the notebook to make it stay connected
function ClickConnect(){
console.log("Clicked on star button");
document.querySelector("iron-icon#star-icon").click()
}
setInterval(ClickConnect,60000)
now you can see the blinking of star every minute.
Solution 18 - Python
The most voted answer certainly works for me but it makes the Manage session window popping up again and again.
I've solved that by auto clicking the refresh button using browser console like below
function ClickRefresh(){
console.log("Clicked on refresh button");
document.querySelector("paper-icon-button").click()
}
setInterval(ClickRefresh, 60000)
Feel free to contribute more snippets for this at this gist https://gist.github.com/Subangkar/fd1ef276fd40dc374a7c80acc247613e
Solution 19 - Python
GNU Colab lets you run a standard persistent desktop environment on top of a Colaboratory instance.
Indeed it contains a mechanism to not let machines die of idling.
Here's a video demonstration.
Solution 20 - Python
Okay I've found a nifty solution that will get rid of the
> Active session
pop up automatically. We'll need 2 function for that,
same procedure as earlier. Inspect> console > paste function one by one
> 1
function clickConnect() {
try {
document.querySelector("#top-toolbar > colab-connect-button").shadowRoot.querySelector("#connect").click();
// this also works, if above one doesn't work, comment it and uncomment below one
//document.querySelector("colab-connect-button").shadowRoot.getElementById('connect').click();
setTimeout(clickDismiss, 500);
console.log("Keeping Colab Alive!");
} catch (error) {
console.log(error);
}
}
> 2
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
> 3
async function clickDismiss() {
try {
// click manage session button
document.querySelector("colab-usage-display").shadowRoot.querySelector("paper-button").click();
} catch (error) {
console.log(error);
}
try {
// leave from manage session window
await sleep(1000);
document.querySelector('colab-sessions-dialog').shadowRoot.querySelector('.dismiss').click();
} catch (error) {
console.log(error);
}
try {
// click close button
await sleep(1000);
document.querySelector("paper-tab").querySelector("paper-icon-button").shadowRoot.getElementById('icon').click();
} catch (error) {
console.log(error);
}
}
> 4
setInterval(ClickConnect, 60000);
EDIT:
So if you don't like doing all this stuff manually there is a way to automate all this!
Way_1. Use this Chrome Extension and done
or
Way_2.
- Use Page-Manipulator extension
- Click on it then click add Java Script button > + New + > filename
- Say file name is ColabAlive > Make
- Active Website =
colab.research.google.com
- Matching pages = Recursive
- Then use below code,
// 1
function clickConnect() {
try {
document.querySelector("#top-toolbar > colab-connect-button").shadowRoot.querySelector("#connect").click();
// this also works, if above one doesn't work, comment it and uncomment below one
//document.querySelector("colab-connect-button").shadowRoot.getElementById('connect').click();
setTimeout(clickDismiss, 500);
console.log("Keeping Colab Alive!");
} catch (error) {
console.log(error);
}
}
//2
function sleep(ms) {
return new Promise(resolve => setTimeout(resolve, ms));
}
//3
async function clickDismiss() {
try {
// click manage session button
document.querySelector("colab-usage-display").shadowRoot.querySelector("paper-button").click();
} catch (error) {
console.log(error);
}
try {
// leave from manage session window
await sleep(1000);
document.querySelector('colab-sessions-dialog').shadowRoot.querySelector('.dismiss').click();
} catch (error) {
console.log(error);
}
try {
// click close button
await sleep(1000);
document.querySelector("paper-tab").querySelector("paper-icon-button").shadowRoot.getElementById('icon').click();
} catch (error) {
console.log(error);
}
}
//4
setInterval(clickConnect, 60000);
- Click Active and then reload > done
Credit goes to Oshayr, Albert Einstein and everyone who posted their solution here.
Solution 21 - Python
This solution in the youtube link below worked for me. install the pynput library that allows you to control and monitor input devices.
pip install pynput
Now execute this code on your local machine and place the mouse cursor in an empty cell in the Colab notebook being run.
from pynput.mouse import Controller,Button
import time
mouse = Controller()
while True:
mouse.click(Button.left,1)
print('clicked')
time.sleep(5)
Solution 22 - Python
I would recommend using JQuery (It seems that Co-lab includes JQuery by default).
function ClickConnect(){
console.log("Working");
$("colab-toolbar-button").click();
}
setInterval(ClickConnect,60000);
Solution 23 - Python
I have a problem with these javascript functions:
function ClickConnect(){
console.log("Clicked on connect button");
document.querySelector("colab-connect-button").click()
}
setInterval(ClickConnect,60000)
They print the "Clicked on connect button" on the console before the button is actually clicked. As you can see from different answers in this thread, the id of the connect button has changed a couple of times since Google Colab was launched. And it could be changed in the future as well. So if you're going to copy an old answer from this thread it may say "Clicked on connect button" but it may actually not do that. Of course if the clicking won't work it will print an error on the console but what if you may not accidentally see it? So you better do this:
function ClickConnect(){
document.querySelector("colab-connect-button").click()
console.log("Clicked on connect button");
}
setInterval(ClickConnect,60000)
And you'll definitely see if it truly works or not.
Solution 24 - Python
function ClickConnect()
{
console.log("Working....");
document.querySelector("paper-button#comments").click()
}
setInterval(ClickConnect,600)
this worked for me but use wisely
happy learning :)
Solution 25 - Python
the following LATEST solution works for me:
function ClickConnect(){
colab.config
console.log("Connnect Clicked - Start");
document.querySelector("#top-toolbar > colab-connect-button").shadowRoot.querySelector("#connect").click();
console.log("Connnect Clicked - End");
};
setInterval(ClickConnect, 60000)
Solution 26 - Python
The javascript below works for me. Credits to @artur.k.space.
function ColabReconnect() {
var dialog = document.querySelector("colab-dialog.yes-no-dialog");
var dialogTitle = dialog && dialog.querySelector("div.content-area>h2");
if (dialogTitle && dialogTitle.innerText == "Runtime disconnected") {
dialog.querySelector("paper-button#ok").click();
console.log("Reconnecting...");
} else {
console.log("ColabReconnect is in service.");
}
}
timerId = setInterval(ColabReconnect, 60000);
In the Colab notebook, click on Ctrl + Shift + the i
key simultaneously. Copy and paste the script into the prompt line. Then hit Enter
before closing the editor.
By doing so, the function will check every 60 seconds to see if the onscreen connection dialog is shown, and if it is, the function would then click the ok
button automatically for you.
Solution 27 - Python
Just run the code below after the cell you want to run to save from data loss.
!python
Also to exit from this mode, write
exit()
Solution 28 - Python
Well I am not a python guy nor I know what is the actual use of this 'Colab', I use it as a build system lol. And I used to setup ssh forwarding in it then put this code and just leave it running and yeah it works.
import getpass
authtoken = getpass.getpass()
Solution 29 - Python
This code keep clicking "Refresh folder" in the file explorer pane.
function ClickRefresh(){
console.log("Working");
document.querySelector("[icon='colab:folder-refresh']").click()
}
const myjob = setInterval(ClickRefresh, 60000)
Solution 30 - Python
Try this to avoid all the annoying dialog boxes appearing while you work when trying to simulate the click on the toolbar connect button every minute. you can just copy paste this to your console, call the method and you can work on your notebook.
function connectRefresher() {
window.ConnectButtonIntervalId = setInterval(function ConnectButton(){
console.log("connected");
document.querySelector("#top-toolbar > colab-connect-button").shadowRoot.querySelector("#connect").click();
document.querySelector("colab-sessions-dialog").shadowRoot.querySelector("#footer > div > paper-button").click();
console.log("closed the dialog!!");
},60000);
}
function clearRefresher() {
console.log("clear Interval called !!");
clearInterval(window.ConnectButtonIntervalId);
}
connectRefresher(); //to connect the refresher
clearRefresher(); //to disconnect the refresher
Solution 31 - Python
You can also use Python to press the arrow keys. I added a little bit of randomness in the following code as well.
from pyautogui import press, typewrite, hotkey
import time
from random import shuffle
array = ["left", "right", "up", "down"]
while True:
shuffle(array)
time.sleep(10)
press(array[0])
press(array[1])
press(array[2])
press(array[3])
Solution 32 - Python
I was looking for a solution until I found a Python3 that randomly moves the mouse back and forth and clicks, always on the same place, but that's enough to fool Colab into thinking I'm active on the notebook and not disconnect.
import numpy as np
import time
import mouse
import threading
def move_mouse():
while True:
random_row = np.random.random_sample()*100
random_col = np.random.random_sample()*10
random_time = np.random.random_sample()*np.random.random_sample() * 100
mouse.wheel(1000)
mouse.wheel(-1000)
mouse.move(random_row, random_col, absolute=False, duration=0.2)
mouse.move(-random_row, -random_col, absolute=False, duration = 0.2)
mouse.LEFT
time.sleep(random_time)
x = threading.Thread(target=move_mouse)
x.start()
You need to install the needed packages: sudo -H pip3 install <package_name>
You just need to run it (in your local machine) with sudo
(as it takes control of the mouse) and it should work, allowing you to take full advantage of Colab's 12h sessions.
Credits: For those using Colab (Pro): Preventing Session from disconnecting due to inactivity
Solution 33 - Python
var startColabHandler = function startColabHandler(interval = 60000, enableConnectButton = false) {
console.log("colabHandler - configure - start: " + new Date());
var colabClick = function colabClick() {
console.log("colabHandler - click - start: " + new Date());
if (enableConnectButton === true) {
var button1 = document.querySelector("#top-toolbar > colab-connect-button").shadowRoot.querySelector("#connect");
if (button1) {
button1.click()
}
}
button2 = document.querySelector("html body colab-dialog.yes-no-dialog paper-dialog div.buttons paper-button#ok");
if (button2) {
button2.click()
}
console.log("colabHandler - click - end: " + new Date());
};
var intervalId = setInterval(colabClick, interval);
window.stopColabHandler = function stopColabHandler() {
console.log("colabHandler - stop - start: " + new Date());
clearInterval(intervalId);
console.log("colabHandler - stop - start: " + new Date());
};
console.log("colabHandler - configure - end: " + new Date());
};
Solution 34 - Python
function ClickConnect(){
console.log("Clicked on connect button");
document.querySelector("connect").click() // Change id here
}
setInterval(ClickConnect,60000)
Try above code it worked for me:)
Solution 35 - Python
prevent using Ipywidgets.
Ipywidgets and similar packages that they prepare Buttons, text fields, etc, complete a cell after a run in Jupiter or Colab. colab saves the last time you ran a cell and if you train your model with these packages then it thinks that you are idle so after 30 minutes it will make you disconnect.