Is there any way to flush output from PL/SQL in Oracle?

I have an SQL script that is called from within a shell script and takes a long time to run. It currently contains dbms_output.put_line statements at various points. The output from these print statements appear in the log files, but only once the script has completed.

Is there any way to ensure that the output appears in the log file as the script is running?

Not really. The way DBMS_OUTPUT works is this: Your PL/SQL block executes on the database server with no interaction with the client. So when you call PUT_LINE, it is just putting that text into a buffer in memory on the server. When your PL/SQL block completes, control is returned to the client (I'm assuming SQLPlus in this case); at that point the client gets the text out of the buffer by calling GET_LINE, and displays it.

So the only way you can make the output appear in the log file more frequently is to break up a large PL/SQL block into multiple smaller blocks, so control is returned to the client more often. This may not be practical depending on what your code is doing.

Other alternatives are to use UTL_FILE to write to a text file, which can be flushed whenever you like, or use an autonomous-transaction procedure to insert debug statements into a database table and commit after each one.

If it is possible to you, you should replace the calls to dbms_output.put_line by your own function.

Here is the code for this function WRITE_LOG -- if you want to have the ability to choose between 2 logging solutions:

write logs to a table in an autonomous transaction

CREATE OR REPLACE PROCEDURE to_dbg_table(p_log varchar2)
  -- table mode: 
  -- requires
  -- CREATE TABLE dbg (u varchar2(200)   --- username
  --                 , d timestamp       --- date
  --                 , l varchar2(4000)  --- log 
  -- );
AS
   pragma autonomous_transaction;
BEGIN
  insert into dbg(u, d, l) values (user, sysdate, p_log);
  commit;
END to_dbg_table;
/

or write directly to the DB server that hosts your database

This uses the Oracle directory TMP_DIR

CREATE OR REPLACE PROCEDURE to_dbg_file(p_fname varchar2, p_log varchar2)
  -- file mode: 
  -- requires
--- CREATE OR REPLACE DIRECTORY TMP_DIR as '/directory/where/oracle/can/write/on/DB_server/';
AS
  l_file utl_file.file_type;
BEGIN
  l_file := utl_file.fopen('TMP_DIR', p_fname, 'A');
  utl_file.put_line(l_file, p_log);
  utl_file.fflush(l_file);
  utl_file.fclose(l_file);
END to_dbg_file;
/

WRITE_LOG

Then the WRITE_LOG procedure which can switch between the 2 uses, or be deactivated to avoid performances loss (g_DEBUG:=FALSE).

CREATE OR REPLACE PROCEDURE write_log(p_log varchar2) AS
  -- g_DEBUG can be set as a package variable defaulted to FALSE
  -- then change it when debugging is required
  g_DEBUG boolean := true;
  -- the log file name can be set with several methods...
  g_logfname varchar2(32767) := 'my_output.log';
  -- choose between 2 logging solutions:
  -- file mode: 
  g_TYPE varchar2(7):= 'file';
  -- table mode: 
  --g_TYPE varchar2(7):= 'table';
  -----------------------------------------------------------------
BEGIN
  if g_DEBUG then
    if g_TYPE='file' then
      to_dbg_file(g_logfname, p_log);
    elsif g_TYPE='table' then
      to_dbg_table(p_log);
    end if;
  end if;  
END write_log;
/

And here is how to test the above:

1. Launch this (file mode) from your SQLPLUS:

   BEGIN write_log('this is a test'); for i in 1..100 loop DBMS_LOCK.sleep(1); write_log('iter=' || i); end loop; write_log('test complete'); END; /

2. on the database server, open a shell and

tail -f -n500 /directory/where/oracle/can/write/on/DB_server/my_output.log

Two alternatives:

1. You can insert your logging details in a logging table by using an autonomous transaction. You can query this logging table in another SQLPLUS/Toad/sql developer etc... session. You have to use an autonomous transaction to make it possible to commit your logging without interfering the transaction handling in your main sql script.

2. Another alternative is to use a pipelined function that returns your logging information. See here for an example: http://berxblog.blogspot.com/2009/01/pipelined-function-vs-dbmsoutput.html When you use a pipelined function you don't have to use another SQLPLUS/Toad/sql developer etc... session.

the buffer of DBMS_OUTPUT is read when the procedure DBMS_OUTPUT.get_line is called. If your client application is SQL*Plus, it means it will only get flushed once the procedure finishes.

You can apply the method described in [this SO][1] to write the DBMS_OUTPUT buffer to a file.

[1]: https://stackoverflow.com/questions/1453538/how-to-redirect-the-output-of-dbmsoutput-putline-to-a-file "How to redirect the output of DBMS_OUTPUT.PUT_LINE to a file?"

Set session metadata MODULE and/or ACTION using dbms_application_info().
Monitor with OEM, for example:

Module: ArchiveData
Action: xxx of xxxx

If you have access to system shell from PL/SQL environment you can call netcat:

 BEGIN RUN_SHELL('echo "'||p_msg||'" | nc '||p_host||' '||p_port||' -w 5'); END;

p_msg - is a log message v_host is a host running python script that reads data from socket on port v_port.

I used this design when I wrote aplogr for real-time shell and pl/sql logs monitoring.