I&#39;m in the second week of Professor Andrew Ng&#39;s Machine Learning course through Coursera. We&#39;re working on linear regression and right now I&#39;m dealing with coding the cost function.

The code I&#39;ve written solves the problem correctly but does not pass the submission process and fails the unit test because I have hard coded the values of theta and not allowed for more than two values for theta.

Here&#39;s the code I&#39;ve got so far

    function J = computeCost(X, y, theta)
    
    m = length(y);
    J = 0;
    
    for i = 1:m,
        h = theta(1) + theta(2) * X(i)
        a = h - y(i);
        b = a^2;
        J = J + b;
        end;
    J = J * (1 / (2 * m));
    
    end

the unit test is 

    computeCost( [1 2 3; 1 3 4; 1 4 5; 1 5 6], [7;6;5;4], [0.1;0.2;0.3])

and should produce ans =  7.0175

So I need to add another for loop to iterate over theta, therefore allowing for any number of values for theta, but I&#39;ll be damned if I can wrap my head around how/where.

Can anyone suggest a way I can allow for any number of values for theta within this function?

If you need more information to understand what I&#39;m trying to ask, I will try my best to provide it.

Cost Function, Linear Regression, trying to avoid hard coding theta. Octave.

Here is an exercise from Introductory Statistics with R:

With the rmr data set, plot metabolic rate versus body weight. Fit a linear regression model to the relation. According to the fitted model, what is the predicted metabolic rate for a body weight of 70 kg? Give a 95% confidence interval for the slope of the line.

rmr data set is in the &#39;ISwR&#39; package. It looks like this:

    &gt; rmr
       body.weight metabolic.rate
    1         49.9           1079
    2         50.8           1146
    3         51.8           1115
    4         52.6           1161
    5         57.6           1325
    6         61.4           1351
    7         62.3           1402
    8         64.9           1365
    9         43.1            870
    10        48.1           1372
    11        52.2           1132
    12        53.5           1172
    13        55.0           1034
    14        55.0           1155
    15        56.0           1392
    16        57.8           1090
    17        59.0            982
    18        59.0           1178
    19        59.2           1342
    20        59.5           1027
    21        60.0           1316
    22        62.1           1574
    23        64.9           1526
    24        66.0           1268
    25        66.4           1205
    26        72.8           1382
    27        74.8           1273
    28        77.1           1439
    29        82.0           1536
    30        82.0           1151
    31        83.4           1248
    32        86.2           1466
    33        88.6           1323
    34        89.3           1300
    35        91.6           1519
    36        99.8           1639
    37       103.0           1382
    38       104.5           1414
    39       107.7           1473
    40       110.2           2074
    41       122.0           1777
    42       123.1           1640
    43       125.2           1630
    44       143.3           1708

I know how to calculate the predicted y at a given x but how can I calculate the confidence interval for the slope?

How to calculate the 95% confidence interval for the slope in a linear regression model in R

Everyone else&#39;s functions seem to take formula objects and then do dark magic to them somewhere deep inside and I&#39;m jealous. 

I&#39;m writing a function that fits multiple models. Parts of the formulas for these models remain the same and part change from one model to the next. The clumsy way would be to have the user input the formula parts as character strings, do some character manipulation on them, and then use `as.formula`.

But before I go that route, I just want to make sure that I&#39;m not overlooking some cleaner way of doing it that would allow the function to accept formulas in the standard R format (e.g. extracted from other formula-using objects).

I want something like...

    &gt; LHS &lt;- y~1; RHS &lt;- ~a+b; c(LHS,RHS);
    y ~ a + b
    &gt; RHS2 &lt;- ~c;
    &gt; c(LHS, RHS, RHS2);
    y ~ a + b + c

or...

    &gt; LHS + RHS;
    y ~ a + b
    &gt; LHS + RHS + RHS2;
    y ~ a + b + c

...but unfortunately neither syntax works. Does anybody know if there is something that does? Thanks.

Is there a better alternative than string manipulation to programmatically build formulas?

Here is what I am doing:

    $ python
    Python 2.7.6 (v2.7.6:3a1db0d2747e, Nov 10 2013, 00:42:54) 
    [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
    &gt;&gt;&gt; import statsmodels.api as sm
    &gt;&gt;&gt; statsmodels.__version__
    &#39;0.5.0&#39;
    &gt;&gt;&gt; import numpy 
    &gt;&gt;&gt; y = numpy.array([1,2,3,4,5,6,7,8,9])
    &gt;&gt;&gt; X = numpy.array([1,1,2,2,3,3,4,4,5])
    &gt;&gt;&gt; res_ols = sm.OLS(y, X).fit()
    &gt;&gt;&gt; res_ols.params
    array([ 1.82352941])

I had expected an array with two elements?!?
The intercept and the slope coefficient?

Why do I get only one parameter from a statsmodels OLS fit

I am trying to implement multivariate linear regression in Python using TensorFlow, but have run into some logical and implementation issues. My code throws the following error:

    Attempting to use uninitialized value Variable
    Caused by op u&#39;Variable/read&#39;

Ideally the `weights` output should be `[2, 3]`

    def hypothesis_function(input_2d_matrix_trainingexamples,
                            output_matrix_of_trainingexamples,
                            initial_parameters_of_hypothesis_function,
                            learning_rate, num_steps):
        # calculate num attributes and num examples
        number_of_attributes = len(input_2d_matrix_trainingexamples[0])
        number_of_trainingexamples = len(input_2d_matrix_trainingexamples)
    
        #Graph inputs
        x = []
        for i in range(0, number_of_attributes, 1):
            x.append(tf.placeholder(&quot;float&quot;))
        y_input = tf.placeholder(&quot;float&quot;)
    
        # Create Model and Set Model weights
        parameters = []
        for i in range(0, number_of_attributes, 1):
            parameters.append(
                tf.Variable(initial_parameters_of_hypothesis_function[i]))
    
        #Contruct linear model
        y = tf.Variable(parameters[0], &quot;float&quot;)
        for i in range(1, number_of_attributes, 1):
            y = tf.add(y, tf.multiply(x[i], parameters[i]))
    
        # Minimize the mean squared errors
        loss = tf.reduce_mean(tf.square(y - y_input))
        optimizer = tf.train.GradientDescentOptimizer(learning_rate)
        train = optimizer.minimize(loss)
    
        #Initialize the variables
        init = tf.initialize_all_variables()
    
        # launch the graph
        session = tf.Session()
        session.run(init)
        for step in range(1, num_steps + 1, 1):
            for i in range(0, number_of_trainingexamples, 1):
                feed = {}
                for j in range(0, number_of_attributes, 1):
                    array = [input_2d_matrix_trainingexamples[i][j]]
                    feed[j] = array
                array1 = [output_matrix_of_trainingexamples[i]]
                feed[number_of_attributes] = array1
                session.run(train, feed_dict=feed)
    
        for i in range(0, number_of_attributes - 1, 1):
            print (session.run(parameters[i]))
    
    array = [[0.0, 1.0, 2.0], [0.0, 2.0, 3.0], [0.0, 4.0, 5.0]]
    hypothesis_function(array, [8.0, 13.0, 23.0], [1.0, 1.0, 1.0], 0.01, 200)
    




TensorFlow: &quot;Attempting to use uninitialized value&quot; in variable initialization

I wonder how to add regression line equation and R^2 on the `ggplot`. My code is:

    library(ggplot2)

    df &lt;- data.frame(x = c(1:100))
    df$y &lt;- 2 + 3 * df$x + rnorm(100, sd = 40)
    p &lt;- ggplot(data = df, aes(x = x, y = y)) +
                geom_smooth(method = &quot;lm&quot;, se=FALSE, color=&quot;black&quot;, formula = y ~ x) +
                geom_point()
    p

Any help will be highly appreciated.


Add regression line equation and R^2 on graph

When we have to predict the value of a [categorical](https://en.wikipedia.org/wiki/Categorical_variable) (or discrete) outcome we use [logistic regression](https://en.wikipedia.org/wiki/Logistic_regression). I believe we use [linear regression](https://en.wikipedia.org/wiki/Linear_regression) to also predict the value of an outcome given the input values.

Then, what is the difference between the two methodologies?

What is the difference between linear regression and logistic regression?

I have a set of data and I want to compare which line describes it best (polynomials of different orders, exponential or logarithmic).

I use Python and Numpy and for polynomial fitting there is a function `polyfit()`. But I found no such functions for exponential and logarithmic fitting. 

Are there any? Or how to solve it otherwise?

How to do exponential and logarithmic curve fitting in Python? I found only polynomial fitting

I&#39;m trying hard to add a regression line on a ggplot. I first tried with abline but I didn&#39;t manage to make it work. Then I tried this...

    data = data.frame(x.plot=rep(seq(1,5),10),y.plot=rnorm(50))
    ggplot(data,aes(x.plot,y.plot))+stat_summary(fun.data=mean_cl_normal) +
       geom_smooth(method=&#39;lm&#39;,formula=data$y.plot~data$x.plot)

But it is not working either.

Adding a regression line on a ggplot

I can&#39;t seem to find any python libraries that do multiple regression. The only things I find only do simple regression. I need to regress my dependent variable (y) against several independent variables (x1, x2, x3, etc.).

For example, with this data:

    print &#39;y        x1      x2       x3       x4      x5     x6       x7&#39;
    for t in texts:
        print &quot;{:&gt;7.1f}{:&gt;10.2f}{:&gt;9.2f}{:&gt;9.2f}{:&gt;10.2f}{:&gt;7.2f}{:&gt;7.2f}{:&gt;9.2f}&quot; /
       .format(t.y,t.x1,t.x2,t.x3,t.x4,t.x5,t.x6,t.x7)

(output for above:)

    

          y        x1       x2       x3        x4     x5     x6       x7
       -6.0     -4.95    -5.87    -0.76     14.73   4.02   0.20     0.45
       -5.0     -4.55    -4.52    -0.71     13.74   4.47   0.16     0.50
      -10.0    -10.96   -11.64    -0.98     15.49   4.18   0.19     0.53
       -5.0     -1.08    -3.36     0.75     24.72   4.96   0.16     0.60
       -8.0     -6.52    -7.45    -0.86     16.59   4.29   0.10     0.48
       -3.0     -0.81    -2.36    -0.50     22.44   4.81   0.15     0.53
       -6.0     -7.01    -7.33    -0.33     13.93   4.32   0.21     0.50
       -8.0     -4.46    -7.65    -0.94     11.40   4.43   0.16     0.49
       -8.0    -11.54   -10.03    -1.03     18.18   4.28   0.21     0.55

How would I regress these in python, to get the linear regression formula:

&gt; Y = a1x1 + a2x2 + a3x3 + a4x4 + a5x5 + a6x6 + +a7x7 + c



Multiple linear regression in Python

How can I tell R to use a certain level as reference if I use binary explanatory variables in a regression?

It&#39;s just using some level by default. 

    lm(x ~ y + as.factor(b)) 

with `b {0, 1, 2, 3, 4}`. Let&#39;s say I want to use 3 instead of the zero that is used by R. 

How to force R to use a specified factor level as reference in a regression?

I want to do a linear regression in R using the `lm()` function. My data is an annual time series with one field for year (22 years) and another for state (50 states). I want to fit a regression for each state so that at the end I have a vector of lm responses. I can imagine doing for loop for each state then doing the regression inside the loop and adding the results of each regression to a vector. That does not seem very R-like, however. In SAS I would do a &#39;by&#39; statement and in SQL I would do a &#39;group by&#39;. What&#39;s the R way of doing this?


Linear Regression and group by in R

I&#39;m trying to generate a linear regression on a scatter plot I have generated, however my data is in list format, and all of the examples I can find of using `polyfit` require using `arange`. `arange` doesn&#39;t accept lists though. I have searched high and low about how to convert a list to an array and nothing seems clear. Am I missing something?

Following on, how best can I use my list of integers as inputs to the `polyfit`?

Here is the polyfit example I am following:
    
    import numpy as np
    import matplotlib.pyplot as plt

    x = np.arange(data)
    y = np.arange(data)

    m, b = np.polyfit(x, y, 1)

    plt.plot(x, y, &#39;yo&#39;, x, m*x+b, &#39;--k&#39;)
    plt.show()

Linear regression with matplotlib / numpy

I&#39;m using `linear_model.LinearRegression` from scikit-learn as a predictive model. It works and it&#39;s perfect. I have a problem to evaluate the predicted results using the `accuracy_score` metric.

This is my true Data :

    array([1, 1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0])

**My predicted Data:**

    array([ 0.07094605,  0.1994941 ,  0.19270157,  0.13379635,  0.04654469,
        0.09212494,  0.19952108,  0.12884365,  0.15685076, -0.01274453,
        0.32167554,  0.32167554, -0.10023553,  0.09819648, -0.06755516,
        0.25390082,  0.17248324])

**My code:**

    accuracy_score(y_true, y_pred, normalize=False)

**Error message:** 

    ValueError: Can&#39;t handle mix of binary and continuous target



Accuracy Score ValueError: Can&#39;t Handle mix of binary and continuous target

I have two vectors of data and I&#39;ve put them into `pyplot.scatter()`. Now I&#39;d like to over plot a linear fit to these data. How would I do this? I&#39;ve tried using `scikitlearn` and `np.polyfit()`.

How to overplot a line on a scatter plot in python?

what is the benefit of using Gradient Descent in the linear regression space? looks like the we can solve the problem (finding theta0-n that minimum the cost func) with analytical method so why we still want to use gradient descent to do the same thing? thanks 

why gradient descent when we can solve linear regression analytically

Regression algorithms seem to be working on features represented as numbers. 
For example:

[![simple data without categorical features][1]][1]


This data set doesn&#39;t contain categorical features/variables. It&#39;s quite clear how to do regression on this data and predict price.

----------

  
But now I want to do a regression analysis on data that contain categorical features:

[![data-set with categorical features][2]][2]

There are **5** features: `District`, `Condition`, `Material`, `Security`, `Type`

  [1]: http://i.stack.imgur.com/deFE1.jpg
  [2]: http://i.stack.imgur.com/gLRrh.jpg

----------

How can I do a regression on this data? Do I have to transform all the string/categorical data to numbers manually? I mean if I have to create some encoding rules and according to that rules transform all data to numeric values. 

Is there any simple way to transform string data to numbers without having to create my own encoding rules manually? Maybe there are some libraries in **Python** that can be used for that? Are there some risks that the regression model will be somehow incorrect due to &quot;bad encoding&quot;?

Linear regression analysis with string/categorical features (variables)?

    def gradient(X_norm,y,theta,alpha,m,n,num_it):
        temp=np.array(np.zeros_like(theta,float))
        for i in range(0,num_it):
            h=np.dot(X_norm,theta)
            #temp[j]=theta[j]-(alpha/m)*(  np.sum( (h-y)*X_norm[:,j][np.newaxis,:] )  )
            temp[0]=theta[0]-(alpha/m)*(np.sum(h-y))
            temp[1]=theta[1]-(alpha/m)*(np.sum((h-y)*X_norm[:,1]))
            theta=temp
        return theta
   
   

    X_norm,mean,std=featureScale(X)
    #length of X (number of rows)
    m=len(X)
    X_norm=np.array([np.ones(m),X_norm])
    n,m=np.shape(X_norm)
    num_it=1500
    alpha=0.01
    theta=np.zeros(n,float)[:,np.newaxis]
    X_norm=X_norm.transpose()
    theta=gradient(X_norm,y,theta,alpha,m,n,num_it)
    print theta

My theta from the above code is `100.2 100.2`, but it should be `100.2 61.09` in matlab which is correct.

gradient descent using python and numpy

As an R user, I wanted to also get up to speed on scikit. 

Creating a linear regression model(s) is fine, but can&#39;t seem to find a reasonable way to get a **standard summary of regression output.** 

Code example:

    # Linear Regression
    import numpy as np
    from sklearn import datasets
    from sklearn.linear_model import LinearRegression

    # Load the diabetes datasets
    dataset = datasets.load_diabetes()

    # Fit a linear regression model to the data
    model = LinearRegression()
    model.fit(dataset.data, dataset.target)
    print(model)

    # Make predictions
    expected = dataset.target
    predicted = model.predict(dataset.data)

    # Summarize the fit of the model
    mse = np.mean((predicted-expected)**2)
    print model.intercept_, model.coef_, mse, 
    print(model.score(dataset.data, dataset.target))

Issues:

 - seems like the **intercept** and **coef** are built into the model, and I just type `print` (second to last line) to see them.
 - What about **all the other standard regression output like R^2, adjusted R^2, p values, etc.** If I read the examples correctly, seems like you have to write a function/equation for each of these and then print it. 
 - So, is there no standard summary output for lin. reg. models?
 - Also, in my printed array of outputs of coefficients, **there are no variable names associated with each of these? I just get the numeric array.** Is there a way to print these where I get an output of the coefficients and the variable they go with? 

My printed output:

    LinearRegression(copy_X=True, fit_intercept=True, normalize=False)
    152.133484163 [ -10.01219782 -239.81908937  519.83978679  324.39042769 -792.18416163
      476.74583782  101.04457032  177.06417623  751.27932109   67.62538639] 2859.69039877
    0.517749425413

Notes: Started off with Linear, Ridge and Lasso. I have gone through the examples. Below is for the basic OLS. 

All Linear Regression Solutions on Gang of Coders

Total of 19 Linear Regression Solutions