If you used `read.table()` (or one of it&#39;s ilk, e.g. `read.csv()`) then the easy fix is to change the call to:

    read.table(file = &quot;foo.txt&quot;, row.names = 1, ....)

where `....` are the other arguments you needed/used. The `row.names` argument takes the column number of the data file from which to take the row names. It need not be the first column. See `?read.table` for details/info.

If you already have the data in R and can&#39;t be bothered to re-read it, or it came from another route, just set the `rownames` attribute and remove the first variable from the object (assuming `obj` is your object)

    rownames(obj) &lt;- obj[, 1]  ## set rownames
    obj &lt;- obj[, -1]           ## remove the first variable

See `?read.table`. Basically, when you use `read.table`, you specify a number indicating the column:

    ##Row names in the first column
    read.table(filname.txt, row.names=1)

Since the last update of Google Chrome, a screen ruler shows up when inspecting an element. Does anyone know how to disable this ruler?

Google Chrome disable screen ruler

On my server 2 days ago my `tmp_table_size` = `max_heap_table_size(16M)`. 

I made a cron job that runs once an hour and generates a report starting from : `created_tmp_disk_tables`, `created_tmp_files`, `created_tmp_tables`

In my report : `created_tmp_disk_tables`+`created_tmp_files`+`created_tmp_tables`=100% of my temporary data

With that : 

1. with `tmp_table_size`=`max_heap_table_size`=`16M` the report showed me the next average report:
- 27.37% (created_tmp_disk_tables) 
- 1.16% (created_tmp_files)
- 71.48% (created_tmp_tables)

How can I optimize these results?

2. with `tmp_table_size`=`max_heap_table_size`=`20M` in the first hour :

- 23.48% (created_tmp_disk_tables) 
- 32.44% (created_tmp_files)
- 44.07% (created_tmp_tables)

After 7 hours (from restart): 

- 21.70% (created_tmp_disk_tables) 
- 33.75% (created_tmp_files)
- 44.55% (created_tmp_tables)

It&#39;s not what I expected. 

- disk tables decreased from `27.37%` to `21.70%` -&gt; expected much more
- temporary files rise form `1.16%` to `33.75%`  -&gt; why ?
- memory tables decreased from `71.48%` to `44.55%` -&gt; strange; expected to rise

Mysql tmp_table_size max_heap_table_size

I have a `.txt` file that contains row names. However, R set the row names as the first column.

Specifying row names when reading in a file

<p>I have a <code>.txt</code> file that contains row names. However, R set the row names as the first column.</p>


I am aware that one can merge (join) two `data.table` with the `merge` function or the `[.data.table` function.  However, if I have say 10, `data.table`s and want to use `do.call` to merge them all together, is there a function that would do this? Currently I resort to `do.call(cbind, ...)` which works for only very special cases.

Merging multiple data.tables

From a dataframe like this

    test &lt;- data.frame(&#39;id&#39;= rep(1:5,2), &#39;string&#39;= LETTERS[1:10])
    test &lt;- test[order(test$id), ]
    rownames(test) &lt;- 1:10

    &gt; test
        id string
     1   1      A
     2   1      F
     3   2      B
     4   2      G
     5   3      C
     6   3      H
     7   4      D
     8   4      I
     9   5      E
     10  5      J

I want to create a new one with the first row of each id / string pair. If sqldf accepted R code within it, the query could look like this:

    res &lt;- sqldf(&quot;select id, min(rownames(test)), string 
                  from test 
                  group by id, string&quot;)

    &gt; res
        id string
     1   1      A
     3   2      B
     5   3      C
     7   4      D
     9   5      E

Is there a solution short of creating a new column like

    test$row &lt;- rownames(test)

and running the same sqldf query with min(row)?

Select the first row by group

When passing missing values to ggplot, it&#39;s very kind, and warns us that they are present. This is acceptable in an interactive session, but when writing reports, you do not the output get cluttered with warnings, especially if there&#39;s many of them. Below example has one label missing, which produces a warning.

    library(ggplot2)
    library(reshape2)
    mydf &lt;- data.frame(
      species = sample(c(&quot;A&quot;, &quot;B&quot;), 100, replace = TRUE), 
      lvl = factor(sample(1:3, 100, replace = TRUE))
    )
    labs &lt;- melt(with(mydf, table(species, lvl)))
    names(labs) &lt;- c(&quot;species&quot;, &quot;lvl&quot;, &quot;value&quot;)
    labs[3, &quot;value&quot;] &lt;- NA
    ggplot(mydf, aes(x = species)) + 
       stat_bin() + 
       geom_text(data = labs, aes(x = species, y = value, label = value, vjust = -0.5)) +
       facet_wrap(~ lvl)

![enter image description here][1]

If we wrap `suppressWarnings` around the last expression, we get a summary of how many warnings there were. For the sake of argument, let&#39;s say that this isn&#39;t acceptable (but is indeed very honest and correct). How to (completely) suppress warnings when printing a ggplot2 object?

  [1]: http://i.stack.imgur.com/l9AnH.png

How to suppress warnings when plotting with ggplot

I&#39;m using ggplot and have two graphs that I want to display on top of each other.  I used `grid.arrange` from gridExtra to stack them.  The problem is I want the left edges of the graphs to align as well as the right edges regardless of axis labels.  (the problem arises because the labels of one graph are short while the other is long).

**The Question:**   
How can I do this?  I am not married to grid.arrange but the ggplot2 is a must.  

**What I&#39;ve tried:**   
I tried playing with widths and heights as well as ncol and nrow to make a 2 x 2 grid and place the visuals in opposite corners and then play with the widths but I couldn&#39;t get the visuals in opposite corners.



    require(ggplot2);require(gridExtra)
    A &lt;- ggplot(CO2, aes(x=Plant)) + geom_bar() +coord_flip() 
    B &lt;- ggplot(CO2, aes(x=Type)) + geom_bar() +coord_flip() 
    grid.arrange(A, B, ncol=1)

![enter image description here][1]


  [1]: http://i.stack.imgur.com/gvnx5.png

Left align two graph edges (ggplot)

I am plotting a graph with a categorical variable on the x axis and a numerical variable on the y axis.

For the x axis, given that there are many data points, the default text formatting causes the label for each tick mark to overlap with other labels. How do I (a) change the font size for my axis text and (b) change the orientation of the text so that the text is perpendicular to the axis?


Changing font size and direction of axes text in ggplot2

I have a CSV file, here is a sample of what it looks like:

    Year:  Dec: Jan:
    1      50   60
    2      25   50
    3      30   30
    4      40   20
    5      10   10

I know how to read the file in and print each column (for ex. - `[&#39;Year&#39;, &#39;1&#39;, &#39;2&#39;, &#39;3&#39;, etc]`). But what I actually want to do is read the rows, which would be like this `[&#39;Year&#39;, &#39;Dec&#39;, &#39;Jan&#39;]` and then `[&#39;1&#39;, &#39;50&#39;, &#39;60&#39;]` and so on. 

And then I would like to store those numbers `[&#39;1&#39;, &#39;50&#39;, &#39;60&#39;]` into variables so I can total them later for ex.:

`Year_1 = [&#39;50&#39;, &#39;60&#39;]`. Then I can do `sum(Year_1) = 110`.

How would I go about doing that in Python 3?

Reading rows from a CSV file in Python

I am new to Python. I need to write some data from my program to a spreadsheet. I&#39;ve searched online and there seem to be many packages available (xlwt, XlsXcessive, openpyxl). Others suggest to write to a .csv file (never used CSV and don&#39;t really understand what it is).

The program is very simple. I have two lists (float) and three variables (strings). I don&#39;t know the lengths of the two lists and they probably won&#39;t be the same length.

I want the layout to be as in the picture below:

![Layout sample][1]


  [1]: http://i.stack.imgur.com/Ih3fa.png

The pink column will have the values of the first list and the green column will have the values of the second list.

So what&#39;s the best way to do this?

P.S. I am running Windows 7 but I won&#39;t necessarily have Office installed on the computers running this program.


    import xlwt

    x=1
    y=2
    z=3

    list1=[2.34,4.346,4.234]

    book = xlwt.Workbook(encoding=&quot;utf-8&quot;)

    sheet1 = book.add_sheet(&quot;Sheet 1&quot;)

    sheet1.write(0, 0, &quot;Display&quot;)
    sheet1.write(1, 0, &quot;Dominance&quot;)
    sheet1.write(2, 0, &quot;Test&quot;)
    
    sheet1.write(0, 1, x)
    sheet1.write(1, 1, y)
    sheet1.write(2, 1, z)

    sheet1.write(4, 0, &quot;Stimulus Time&quot;)
    sheet1.write(4, 1, &quot;Reaction Time&quot;)

    i=4

    for n in list1:
        i = i+1
        sheet1.write(i, 0, n)
    


    book.save(&quot;trial.xls&quot;)

I wrote this using all your suggestions. It gets the job done but it can be slightly improved.

How do I format the cells created in the for loop (list1 values) as scientific or number?

I do not want to truncate the values. The actual values used in the program would have around 10 digits after the decimal.

Writing to an Excel spreadsheet

I would like to know the following;

1. Why the given non-working example doesn&#39;t work.
2. If there are any other cleaner methods than those given in working example.

*Non-working example*

    &gt; ids=(1 2 3 4);echo ${ids[*]// /|}
    1 2 3 4
    &gt; ids=(1 2 3 4);echo ${${ids[*]}// /|}
    -bash: ${${ids[*]}// /|}: bad substitution
    &gt; ids=(1 2 3 4);echo ${&quot;${ids[*]}&quot;// /|}
    -bash: ${&quot;${ids[*]}&quot;// /|}: bad substitution

*Working example*

    &gt; ids=(1 2 3 4);id=&quot;${ids[@]}&quot;;echo ${id// /|}
    1|2|3|4
    &gt; ids=(1 2 3 4); lst=$( IFS=&#39;|&#39;; echo &quot;${ids[*]}&quot; ); echo $lst
    1|2|3|4

In context, the delimited string to be used in a [sed][1] command for further parsing.


  [1]: https://www.gnu.org/software/sed/manual/sed.html

Converting a Bash array into a delimited string

My source data is in a TSV file, 6 columns and greater than 2 million rows.

Here&#39;s what I&#39;m trying to accomplish:

1. I need to read the data in 3 of the columns (3, 4, 5) in this source file
2. The fifth column is an integer. I need to use this integer value to duplicate a row entry with using the data in the third and fourth columns (by the number of integer times).
3. I want to write the output of #2 to an output file in CSV format.

Below is what I came up with.  

My question: is this an efficient way to do it?  It seems like it might be intensive when attempted on 2 million rows.

First, I made a sample tab separate file to work with, and called it &#39;sample.txt&#39;.  It&#39;s basic and only has four rows:

    Row1_Column1	Row1-Column2	Row1-Column3	Row1-Column4	2	Row1-Column6
    Row2_Column1	Row2-Column2	Row2-Column3	Row2-Column4	3	Row2-Column6
    Row3_Column1	Row3-Column2	Row3-Column3	Row3-Column4	1	Row3-Column6
    Row4_Column1	Row4-Column2	Row4-Column3	Row4-Column4	2	Row4-Column6

then I have this code:

    import csv 
    
    with open(&#39;sample.txt&#39;,&#39;r&#39;) as tsv:
        AoA = [line.strip().split(&#39;\t&#39;) for line in tsv]
    
    for a in AoA:
        count = int(a[4])
        while count &gt; 0:
            with open(&#39;sample_new.csv&#39;, &#39;a&#39;, newline=&#39;&#39;) as csvfile:
                csvwriter = csv.writer(csvfile, delimiter=&#39;,&#39;)
                csvwriter.writerow([a[2], a[3]])
            count = count - 1


reading and parsing a TSV file, then manipulating it for saving as CSV (*efficiently*)

I have a long list of lists of the following form ---

    a = [[1.2,&#39;abc&#39;,3],[1.2,&#39;werew&#39;,4],........,[1.4,&#39;qew&#39;,2]]

i.e. the values in the list are of different types -- float,int, strings.How do I write it into a csv file so that my output csv file looks like

    1.2,abc,3
    1.2,werew,4
    .
    .
    .
    1.4,qew,2




Writing a Python list of lists to a csv file

I have a Pandas Dataframe as below:

          itm Date                  Amount 
    67    420 2012-09-30 00:00:00   65211
    68    421 2012-09-09 00:00:00   29424
    69    421 2012-09-16 00:00:00   29877
    70    421 2012-09-23 00:00:00   30990
    71    421 2012-09-30 00:00:00   61303
    72    485 2012-09-09 00:00:00   71781
    73    485 2012-09-16 00:00:00     NaN
    74    485 2012-09-23 00:00:00   11072
    75    485 2012-09-30 00:00:00  113702
    76    489 2012-09-09 00:00:00   64731
    77    489 2012-09-16 00:00:00     NaN

When I try to apply a function to the Amount column, I get the following error:

    ValueError: cannot convert float NaN to integer

I have tried applying a function using .isnan from the Math Module
I have tried the pandas .replace attribute
I tried the .sparse data attribute from pandas 0.9
I have also tried if NaN == NaN statement in a function.
I have also looked at this article https://stackoverflow.com/questions/8161836/how-do-i-replace-na-values-with-zeros-in-r whilst looking at some other articles. 
All the methods I have tried have not worked or do not recognise NaN.
Any Hints or solutions would be appreciated. 

How to replace NaN values by Zeroes in a column of a Pandas Dataframe?

Suppose I have a `df` which has columns of `&#39;ID&#39;, &#39;col_1&#39;, &#39;col_2&#39;`. And I define a function :

 `f = lambda x, y : my_function_expression`.


Now I want to apply the `f` to `df`&#39;s two columns `&#39;col_1&#39;, &#39;col_2&#39;` to element-wise calculate a new column `&#39;col_3&#39;` , somewhat like :

    df[&#39;col_3&#39;] = df[[&#39;col_1&#39;,&#39;col_2&#39;]].apply(f)  
    # Pandas gives : TypeError: (&#39;&lt;lambda&gt;() takes exactly 2 arguments (1 given)&#39;

  

How to do ?

********* Add detail sample as below ********

    import pandas as pd
    
    df = pd.DataFrame({&#39;ID&#39;:[&#39;1&#39;,&#39;2&#39;,&#39;3&#39;], &#39;col_1&#39;: [0,2,3], &#39;col_2&#39;:[1,4,5]})
    mylist = [&#39;a&#39;,&#39;b&#39;,&#39;c&#39;,&#39;d&#39;,&#39;e&#39;,&#39;f&#39;]
    
    def get_sublist(sta,end):
    	return mylist[sta:end+1]
        
    #df[&#39;col_3&#39;] = df[[&#39;col_1&#39;,&#39;col_2&#39;]].apply(get_sublist,axis=1)
    # expect above to output df as below 

      ID  col_1  col_2            col_3
    0  1      0      1       [&#39;a&#39;, &#39;b&#39;]
    1  2      2      4  [&#39;c&#39;, &#39;d&#39;, &#39;e&#39;]
    2  3      3      5  [&#39;d&#39;, &#39;e&#39;, &#39;f&#39;]

How to apply a function to two columns of Pandas dataframe

When deleting a column in a DataFrame I use:

    del df[&#39;column_name&#39;]

And this works great. Why can&#39;t I use the following?

    del df.column_name

Since it is possible to access the column/Series as `df.column_name`, I expected this to work.




Delete a column from a Pandas DataFrame

I have this `DataFrame` and want only the records whose `EPS` column is not `NaN`:

    &gt;&gt;&gt; df
                     STK_ID  EPS  cash
    STK_ID RPT_Date                   
    601166 20111231  601166  NaN   NaN
    600036 20111231  600036  NaN    12
    600016 20111231  600016  4.3   NaN
    601009 20111231  601009  NaN   NaN
    601939 20111231  601939  2.5   NaN
    000001 20111231  000001  NaN   NaN

...i.e. something like `df.drop(....)` to get this resulting dataframe:

                      STK_ID  EPS  cash
    STK_ID RPT_Date                   
    600016 20111231  600016  4.3   NaN
    601939 20111231  601939  2.5   NaN

How do I do that?

How to drop rows of Pandas DataFrame whose value in a certain column is NaN

I am creating a dataframe using this code:

    df &lt;- data.frame(dbGetQuery(con, paste(&#39;select * from test&#39;)))

Which results in this:

        UID      BuildingCode   AccessTime
    1   123456   BUILD-1        2014-06-16 07:00:00
    2   364952   BUILD-2        2014-06-15 08:00:00
    3    95865   BUILD-1        2014-06-06 09:50:00

I am then trying to remove the row names (1, 2, 3, etc) as suggested [here][1] by using this code:

    rownames(df) &lt;- NULL

But then when I print out `df` it still displays the row names. Is there a way to not include the row names when creating the data frame? I found a suggestion about `row.name = FALSE` but when I tried it I just got errors (I might have placed it in the wrong place).

**EDIT:** What I want to do is convert the dateframe to a HTML table and I don&#39;t want the row name to be present in the table.


  [1]: https://stackoverflow.com/questions/12117629/how-to-avoid-writing-a-row-names-column-when-saving-a-data-frame-using-the-xlsx

Removing display of row names from data frame

I have a data frame like this: 

    df
                  VALUE              ABS_CALL DETECTION P-VALUE    
        1007_s_at &quot;957.729231881542&quot; &quot;P&quot;      &quot;0.00486279317241156&quot;
        1053_at   &quot;320.632701283368&quot; &quot;P&quot;      &quot;0.0313356324173416&quot; 
        117_at    &quot;429.842323161046&quot; &quot;P&quot;      &quot;0.0170004527476119&quot; 
        121_at    &quot;2395.7364289242&quot;  &quot;P&quot;      &quot;0.0114473584876183&quot; 
        1255_g_at &quot;116.493632746934&quot; &quot;A&quot;      &quot;0.39799368200131&quot;   
        1294_at   &quot;739.927122116896&quot; &quot;A&quot;      &quot;0.0668649772942343&quot; 

I want to convert the row names into the first column. Currently I use something  like this to make row names as the first column:

      d &lt;- df
      names &lt;- rownames(d)
      rownames(d) &lt;- NULL
      data &lt;- cbind(names,d)

Is there a single line to do this?
   

Convert row names into first column

I have a dataframe like this one:

    In [10]: df
    Out[10]: 
             Column 1
    foo              
    Apples          1
    Oranges         2
    Puppies         3
    Ducks           4

How to remove `index name` `foo` from that dataframe?
The desired output is like this:


    In [10]: df
    Out[10]: 
             Column 1             
    Apples          1
    Oranges         2
    Puppies         3
    Ducks           4

Content Type	Original Author	Original Content on Stackoverflow
Question	fokos	View Question on Stackoverflow
Solution 1 - R	Gavin Simpson	View Answer on Stackoverflow
Solution 2 - R	csgillespie	View Answer on Stackoverflow

Specifying row names when reading in a file

R Problem Overview

R Solutions

Solution 1 - R

Solution 2 - R

Mysql tmp_table_size max_heap_table_size

Google Chrome disable screen ruler

Attributions