From the [documentation](http://mongoosejs.com/docs/guide.html#id):

&gt; Mongoose assigns each of your schemas an id virtual getter by default
&gt; which returns the documents _id field cast to a string, or in the case
&gt; of ObjectIds, its hexString.

So, basically, the `id` getter returns a string representation of the document&#39;s `_id` (which is added to all MongoDB documents by default and have a default type of `ObjectId`).

Regarding what&#39;s better for referencing, that depends entirely on the context (i.e., do you want an `ObjectId` or a `string`). For example, if comparing `id`&#39;s, the string is probably better, as `ObjectId`&#39;s won&#39;t pass an equality test unless they are the same instance (regardless of what value they represent).

I&#39;ve created a time point, but I have been struggling to print it to the terminal.

    #include &lt;iostream&gt;
    #include &lt;chrono&gt;
    
    int main(){
    
    	//set time_point to current time
    	std::chrono::time_point&lt;std::chrono::system_clock,std::chrono::nanoseconds&gt; time_point;
    	time_point = std::chrono::system_clock::now();
    
    	//print the time
    	//...
    	
    	return 0;
    }

The only documentation I can find that prints a time_point is found here:
http://en.cppreference.com/w/cpp/chrono/time_point

however, I&#39;m not even able to create a time_t based on my time_point(like the example).

    std::time_t now_c = std::chrono::system_clock::to_time_t(time_point); //does not compile

**Error:**

    /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/chrono: In instantiation of ‘constexpr std::chrono::time_point&lt;_Clock, _Dur&gt;::time_point(const std::chrono::time_point&lt;_Clock, _Dur2&gt;&amp;) [with _Dur2 = std::chrono::duration&lt;long int, std::ratio&lt;1l, 1000000000l&gt; &gt;; _Clock = std::chrono::system_clock; _Dur = std::chrono::duration&lt;long int, std::ratio&lt;1l, 1000000l&gt; &gt;]’:
    time.cpp:13:69:   required from here
    /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/chrono:540:32: error: no matching function for call to ‘std::chrono::duration&lt;long int, std::ratio&lt;1l, 1000000l&gt; &gt;::duration(std::chrono::time_point&lt;std::chrono::system_clock, std::chrono::duration&lt;long int, std::ratio&lt;1l, 1000000000l&gt; &gt; &gt;::duration)’
    /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/chrono:540:32: note: candidates are:
    /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/chrono:247:14: note: template&lt;class _Rep2, class _Period2, class&gt; constexpr std::chrono::duration::duration(const std::chrono::duration&lt;_Rep2, _Period2&gt;&amp;)
    /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/chrono:247:14: note:   template argument deduction/substitution failed:
    /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/chrono:243:46: error: no type named ‘type’ in ‘struct std::enable_if&lt;false, void&gt;’
    /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/chrono:240:23: note: template&lt;class _Rep2, class&gt; constexpr std::chrono::duration::duration(const _Rep2&amp;)
    /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/chrono:240:23: note:   template argument deduction/substitution failed:
    /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/chrono:236:27: error: no type named ‘type’ in ‘struct std::enable_if&lt;false, void&gt;’
    /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/chrono:234:12: note: constexpr std::chrono::duration&lt;_Rep, _Period&gt;::duration(const std::chrono::duration&lt;_Rep, _Period&gt;&amp;) [with _Rep = long int; _Period = std::ratio&lt;1l, 1000000l&gt;]
    /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/chrono:234:12: note:   no known conversion for argument 1 from ‘std::chrono::time_point&lt;std::chrono::system_clock, std::chrono::duration&lt;long int, std::ratio&lt;1l, 1000000000l&gt; &gt; &gt;::duration {aka std::chrono::duration&lt;long int, std::ratio&lt;1l, 1000000000l&gt; &gt;}’ to ‘const std::chrono::duration&lt;long int, std::ratio&lt;1l, 1000000l&gt; &gt;&amp;’
    /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/chrono:232:12: note: constexpr std::chrono::duration&lt;_Rep, _Period&gt;::duration() [with _Rep = long int; _Period = std::ratio&lt;1l, 1000000l&gt;]
    /usr/lib/gcc/x86_64-redhat-linux/4.7.2/../../../../include/c++/4.7.2/chrono:232:12: note:   candidate expects 0 arguments, 1 provided

How do you print a C++11 time_point?

Currently, I have this line in a Markdown file detailing command output:

    1\. Work (00:10:00)  
        1\. Mail letter (00:05:00, Est. 00:03:00)  
          Send letter to Foo Bar  
    2\. Personal (00:02:00)

However, when I preview the Markdown file, all of the whitespace is disregarded.

&gt; 1\. Work (00:10:00)  
&gt;     1\. Mail letter (00:05:00, Est. 00:03:00)  
&gt;       Send letter to Foo Bar  
&gt; 2\. Personal (00:02:00)

How do I preserve this whitespace?

How do I ensure that whitespace is preserved in Markdown?

What is the difference between `_id` and `id` in mongoose? Which is better for referencing?

What is the difference between id and _id in mongoose?

<p>What is the difference between <code>_id</code> and <code>id</code> in mongoose? Which is better for referencing?</p>


I am writing a JavaScript client to be included on 3rd party sites (think Facebook Like button). It needs to retrieve information from an API that requires basic HTTP authentication. The simplified setup looks like this:

A 3rd party site includes this snippet on their page:


    &lt;script 
    async=&quot;true&quot;
    id=&quot;web-dev-widget&quot;
    data-public-key=&quot;pUbl1c_ap1k3y&quot;
    src=&quot;http://web.dev/widget.js&quot;&gt;
    &lt;/script&gt;


**widget.js** calls the API:

    var el = document.getElementById(&#39;web-dev-widget&#39;),
        user = &#39;token&#39;,
        pass = el.getAttribute(&#39;data-public-key&#39;),
        url = &#39;https://api.dev/&#39;,
        httpRequest = new XMLHttpRequest(),
        handler = function() {
          if (httpRequest.readyState === 4) {
            if (httpRequest.status === 200) {
              console.log(httpRequest.responseText);
            } else {
              console.log(&#39;There was a problem with the request.&#39;, httpRequest);
            }
          }
        };

    httpRequest.open(&#39;GET&#39;, url, true, user, pass);
    httpRequest.onreadystatechange = handler;
    httpRequest.withCredentials = true;
    httpRequest.send();

The API has been configured to respond with appropriate headers:

    Header set Access-Control-Allow-Credentials: true
    Header set Access-Control-Allow-Methods: &quot;GET, OPTIONS&quot;
    Header set Access-Control-Allow-Headers: &quot;origin, authorization, accept&quot;
    SetEnvIf Origin &quot;http(s)?://(.+?\.[a-z]{3})$&quot; AccessControlAllowOrigin=$0
    Header set Access-Control-Allow-Origin %{AccessControlAllowOrigin}e env=AccessControlAllowOrigin

Note that the `Access-Control-Allow-Origin` is set to the `Origin` instead of using a wildcard because I am sending a credentialed request (`withCredentials`).

Everything is now in place to make an asynchronous cross-domain authenticated request, and it works great in Chrome 25 on OS X 10.8.2. In Dev Tools, I can see the network request for the `OPTIONS` request before the `GET` request, and the response comes back as expected.

When testing in Firefox 19, no network requests appear in Firebug to the API, and this error is logged in the console: `NS_ERROR_DOM_BAD_URI: Access to restricted URI denied`

After much digging, I found that [Gecko doesn&#39;t allow the username and password to be directly in a cross-site URI](https://stackoverflow.com/questions/11255173/firefox-wont-send-cross-origin-resource-sharing-pre-flight) according to the comments. I assumed this was from using the optional user and password params to `open()` so I tried the other method of making authenticated requests which is to Base64 encode the credentials and send in an Authorization header:

    // Base64 from http://www.webtoolkit.info/javascript-base64.html
    auth = &quot;Basic &quot; + Base64.encode(user + &quot;:&quot; + pass);
    
    ...
    // after open() and before send()
    httpRequest.setRequestHeader(&#39;Authorization&#39;, auth);

This results in a `401 Unauthorized` response to the `OPTIONS` request which lead to Google searches like, &quot;Why does this work in Chrome and not Firefox!?&quot; That&#39;s when I knew I was in trouble.

Why _does_ it work in Chrome and not Firefox? How can I get the `OPTIONS` request to send and respond consistently?

Why does the preflight OPTIONS request of an authenticated CORS request work in Chrome but not Firefox?

How would I hide the first element of an ng-repeat?

    &lt;div ng-repeat=&quot;item in items&quot; ng-hide=&quot;true&quot;&gt;
    	&lt;div&gt;{{ item.value }}&lt;/div&gt;
    &lt;/div&gt;

This works in that the whole ng-repeat block is hidden, but how would I hide only the first item in items? I want to display it completely differently using more prominent html/etc, so it&#39;s useful to have it in that list of data.

Angular JS hide first element of ng-repeat

How do I click an element in PhantomJS?

    page.evaluate(function() {
        document.getElementById(&#39;idButtonSpan&#39;).click();  
    });

This gives me an error &quot;undefined is not a function...&quot;

If I instead 

     return document.getElementById(&#39;idButtonSpan&#39;);

and then print it,

then it prints [object object], so the element does exist.

The element acts as a button, but it&#39;s actually just a span element, not a submit input.


I was able to get this button click to work with Casper, but Casper had other limitations so I&#39;m back to PhantomJS.

PhantomJS; click an element

I&#39;m using [datatables][1] for displaying server-side data in tables.

I can&#39;t target and style individual cells (`&lt;TD&gt;`) though. I search a bit and found it might be possible with:

    &quot;fnRowCallback&quot;: function( nRow, aData, iDisplayIndex ) {
        ....
    }

... but I&#39;m not quite sure how because I have a few table and not all have the same number of columns and rows. I want to give common class to all `TDs` of a &#39;column&#39;. 

  [1]: http://www.datatables.net

Giving custom classes to each TD for styling - Datatables &amp; jQuery

I am creating a light box in pure JavaScript. For that I am making an overlay. I want to add this overlay to body but I also want to keep the content on the page. My current code adds the overlay div but it also removes the current contents in body. **How to add div element and keep contents on body?**

    var el = document.getElementById(&#39;element&#39;);
	var body = document.getElementsByTagName(&#39;body&#39;);
	el.innerHTML = &#39;&lt;p&gt;&lt;a id=&quot;clickme&quot; href=&quot;#&quot;&gt;Click me&lt;/a&gt;&lt;/p&gt;&#39;;
	document.getElementById(&#39;clickme&#39;).onclick = function (e) {
		e.preventDefault();
		document.body.innerHTML = &#39;&lt;div style=&quot;position:absolute;width:100%;height:100%;opacity:0.3;z-index:100;background:#000;&quot;&gt;&lt;/div&gt;&#39;;
	}

Adding div element to body or document in JavaScript

I need to upload an image, and display it, as well as save it so that I don&#39;t lose it when I refresh the localhost. This needs to be done using an &quot;Upload&quot; button, which prompts for a file-selection. 

I am using node.js and express for the server-side code.


How to upload, display and save images using node.js and express

Is it possible to prevent SQL injections in Node.js (preferably with a module) in the same way that PHP had Prepared Statements that protected against them.

If so, how? If not, **what are some examples** that might bypass the code I&#39;ve provided (see below).

----------------
Some Context:

I&#39;m making a web application with a back-end stack consisting of Node.js + MySql using the [node-mysql][1] module. From a usability perspective, the module is great, but it has not yet implemented something akin to PHP&#39;s [Prepared Statements][2] (though I&#39;m aware it is on the [todo][3]).

From my understanding, PHP&#39;s implementation of prepared statements, among other things, [helped greatly][4] in the prevention of SQL injections. I&#39;m worried, though, that my node.js app may be open to similar attacks, [even with the string escaping provided by default][5] (as in the code snippet below).

node-mysql seems to be the most popular mysql connector for node.js, so I was wondering what other people might be doing (if anything) to account for this issue - or if it is even an issue with node.js to begin with (not sure how this wouldn&#39;t be, since user/client-side input is involved).

**Should I switch to [node-mysql-native][6] for the time being, since it does provide prepared statements?** I&#39;m hesitant to do this, because it does not seem to be as active as node-mysql (though that may just mean that it is complete).

Here is a snippet of user registration code, which uses the [sanitizer][7] module, along with node-mysql&#39;s prepared statement-like syntax (which, as I mentioned above, does character escaping), to prevent cross site scripting and sql injections, respectively:

    // Prevent xss
    var clean_user = sanitizer.sanitize(username);

    // assume password is hashed already
    var post = {Username: clean_user, Password: hash};
           
    // This just uses connection.escape() underneath
    var query = connection.query(&#39;INSERT INTO users SET ?&#39;, post,
       function(err, results)
       {
           // Can a Sql injection happen here?
       });


  [1]: https://github.com/felixge/node-mysql
  [2]: http://php.net/manual/en/pdo.prepared-statements.php
  [3]: https://github.com/felixge/node-mysql#todo
  [4]: https://stackoverflow.com/questions/5741187/sql-injection-that-gets-around-mysql-real-escape-string/12118602#12118602
  [5]: https://github.com/felixge/node-mysql#escaping-query-values
  [6]: https://github.com/sidorares/nodejs-mysql-native
  [7]: https://github.com/theSmaw/Caja-HTML-Sanitizer

Preventing SQL injection in Node.js

I would like to use the `grunt-contrib-jasmine` NPM package. It has various dependencies. Part of the dependency graph looks like this:

    ─┬ grunt-contrib-jasmine@0.4.1
     │ ├─┬ grunt-lib-phantomjs@0.2.0
     │ │ ├─┬ phantomjs@1.8.2-2

Unfortunately, there&#39;s a bug in this version `phantomjs` which prevents it from installing correctly on Mac OS X. This is fixed in the latest version.

**How can I get `grunt-lib-phantomjs` to use a newer version of `phantomjs`?**

Some additional context:

* `grunt-contrib-jasmine` explicitly requires version `&quot;~0.2.0&quot;` of `grunt-lib-phantomjs`, which explicitly requires version `&quot;~1.8.1&quot;` of `phantomjs`.
* Adding `phantomjs` to my package&#39;s dependencies first has no effect; both versions are installed and `grunt-contrib-jasmine` still uses the older versions (see: https://stackoverflow.com/questions/11233133/when-installing-a-package-with-npm-can-you-tell-it-to-use-a-different-version-o).

How do I override nested NPM dependency versions?

I have an application which has the usual set of dependencies on third party modules (e.g. &#39;express&#39;) specified in the package.json file under dependencies. E.g.

    &quot;express&quot;     : &quot;3.1.1&quot;
I would like to structure my own code modularly and have a set of local (meaning on the file system i am currently in) modules be installed by the package.json. I know that i can install a local module by running:

    npm install path/to/mymodule
However, I don&#39;t know how to make this happen via the package.json dependencies structure. Using the `--save` option in this command is simply putting `&quot;mymodule&quot;: &quot;0.0.0&quot;` into my package.json (doesn&#39;t reference the filepath location). If i then remove the installed version from node_modules, and try to re-install from the package.json, it fails (because it looks for &quot;mymodule&quot; in the central registry, and doesn&#39;t look locally).

I&#39;m sure the is a way of telling the `&quot;dependencies&quot;: {}` structure that I want it to be installed from a file system path, but don&#39;t know how.

Anyone else had this problem?
Thanks.

how to specify local modules as npm package dependencies

I&#39;m working on a nodejs project for school.
I wasn&#39;t able to install bcrypt with npm so i installed bcrypt-nodejs and the project worked fine yesterday.
But today, when I do a &quot;node app&quot; i have this error :


    /.../node_modules/bcrypt/node_modules/bindings/bindings.js:79
            throw e
                  ^
    Error: /.../node_modules/bcrypt/build/Release/bcrypt_lib.node: invalid ELF header
        at Module.load (module.js:356:32)
        at Function.Module._load (module.js:312:12)
        at Module.require (module.js:364:17)
        at require (module.js:380:17)
        at bindings (/.../node_modules/bcrypt/node_modules/bindings/bindings.js:74:15)
        at Object.&lt;anonymous&gt; (/.../node_modules/bcrypt/bcrypt.js:1:97)
        at Module._compile (module.js:456:26)
        at Object.Module._extensions..js (module.js:474:10)
        at Module.load (module.js:356:32)
        at Function.Module._load (module.js:312:12)
        at Module.require (module.js:364:17)
        at require (module.js:380:17)

my package.json file looks like this:


    {
      &quot;name&quot;: &quot;Supinfarm&quot;,
      &quot;version&quot;: &quot;0.0.0&quot;,
      &quot;env&quot;: {
                  &quot;PYTHON&quot;: &quot;/usr/bin/python2.6&quot;
            },
      &quot;dependencies&quot;: {
        &quot;express&quot;: &quot;3.1.0&quot;,
        &quot;connect-flash&quot;: &quot;*&quot;,
        &quot;jade&quot;: &quot;*&quot;,
        &quot;stylus&quot;: &quot;*&quot;,
        &quot;passport&quot;: &quot;*&quot;,
        &quot;passport-local&quot;: &quot;*&quot;,
        &quot;mongoose&quot;: &quot;*&quot;,
        &quot;bcrypt&quot;: &quot;*&quot;
      }
    }

I&#39;m on Linux ubuntu 10.04 LTS
I&#39;ve tried to find a solution on google without success...
Can somebody help me?


bcrypt invalid elf header when running node app

For my project, I want to keep a mongoose document for groups of organizations, like this:

    var groupSchema = Schema({
      name : { type : String },
      org : { type : Schema.Types.ObjectId, ref : &#39;Organization&#39; },
      ...
      users : [{
        uid : { type : Schema.Types.ObjectId, ref : &#39;User&#39; },
        ...
      }]
    });

I want to prevent the same user from being in the same group twice. To do this, I need to force users.uid to be unique in the users array. I tried stating &#39;unique : true&#39; for uid, but that didn&#39;t work. Is there a way to do this with mongoose or mongoDB without extra queries or splitting the schema?

Edit:
I changed the previous value of uid to
    uid : { type : Schema.Types.ObjectId, ref : &#39;User&#39;, index: {unique: true, dropDups: true} }
But this still doesn&#39;t seem to work.

Edit: Assuming there is no simple way to achieve this, I added an extra query checking if the user is already in the group. This seems to me the simplest way.

Mongoose Unique values in nested array of objects

I have some docs in mongo that looks something like this: 

    {
      _id : ObjectId(&quot;...&quot;),
      &quot;make&quot; : &quot;Nissan&quot;,
      ..
    },
    {
      _id : ObjectId(&quot;...&quot;),
      &quot;make&quot; : &quot;Nissan&quot;,
      &quot;saleDate&quot; :  ISODate(&quot;2013-04-10T12:39:50.676Z&quot;),
      ..
    }

Ideally, I&#39;d like to be able to count, by make, the number of vehicles sold per day.  I&#39;d then like to view either today, or a window such as today through the last seven days.

I was able to accomplish the daily view with some ugly code

    db.inventory.aggregate(
      { $match : { &quot;saleDate&quot; : { $gte: ISODate(&quot;2013-04-10T00:00:00.000Z&quot;), $lt: ISODate(&quot;2013-04-11T00:00:00.000Z&quot;)  } } } ,
      { $group : { _id : { make : &quot;$make&quot;, saleDayOfMonth : { $dayOfMonth : &quot;$saleDate&quot; } }, cnt : { $sum : 1 } } }
    )

Which then yields the results

    {
      &quot;result&quot; : [
        {
          &quot;_id&quot; : {
            &quot;make&quot; : &quot;Nissan&quot;,
            &quot;saleDayOfMonth&quot; : 10
          },
          &quot;cnt&quot; : 2
        },
        {
          &quot;_id&quot; : {
            &quot;make&quot; : &quot;Toyota&quot;,
            &quot;saleDayOfMonth&quot; : 10
          },
          &quot;cnt&quot; : 4
        },
      ],
      &quot;ok&quot; : 1
    }

So that is ok, but I would much prefer to not have to change the two datetime values in the query.  Then, as I mentioned above, I&#39;d like to be able to run this query (again, without having to modify it each time) and see the same results binned by day over the last week.

Oh and here is the sample data I&#39;ve been using for the query

    db.inventory.save({&quot;make&quot; : &quot;Nissan&quot;,&quot;saleDate&quot; :  ISODate(&quot;2013-04-10T12:39:50.676Z&quot;)});
    db.inventory.save({&quot;make&quot; : &quot;Nissan&quot;});
    db.inventory.save({&quot;make&quot; : &quot;Nissan&quot;,&quot;saleDate&quot; :  ISODate(&quot;2013-04-10T11:39:50.676Z&quot;)});
    db.inventory.save({&quot;make&quot; : &quot;Toyota&quot;,&quot;saleDate&quot; :  ISODate(&quot;2013-04-09T11:39:50.676Z&quot;)});
    db.inventory.save({&quot;make&quot; : &quot;Toyota&quot;,&quot;saleDate&quot; :  ISODate(&quot;2013-04-10T11:38:50.676Z&quot;)});
    db.inventory.save({&quot;make&quot; : &quot;Toyota&quot;,&quot;saleDate&quot; :  ISODate(&quot;2013-04-10T11:37:50.676Z&quot;)});
    db.inventory.save({&quot;make&quot; : &quot;Toyota&quot;,&quot;saleDate&quot; :  ISODate(&quot;2013-04-10T11:36:50.676Z&quot;)});
    db.inventory.save({&quot;make&quot; : &quot;Toyota&quot;,&quot;saleDate&quot; :  ISODate(&quot;2013-04-10T11:35:50.676Z&quot;)});

Thanks in advance,
Kevin


MongoDB aggregate within daily grouping

Ran into this (slightly annoying problem) I&#39;m trying to look up all records in a collection and not show (or show) a specific field (score). This is just an example and not reflective of my actual code, but the problem is always reproducible. The fields I am excluding are very large and I&#39;m merely trying to generate a menu of available records.

Commands like 

        players.find({},{score:1})
        players.find({},{score:0})

Always return every field, instead of triggering the exclude/include in mongodb. Am I worried about nothing, since the template potentially can control what data gets rendered to html? Still feels like the data is transferred to the client side regardless; and shows up in the console.

Meteor - collection.find() always returns all fields

I have a problem when querying mongoDB with nested objects notation:

    db.messages.find( { headers : { From: &quot;reservations@marriott.com&quot; } } ).count()
    0
    db.messages.find( { &#39;headers.From&#39;: &quot;reservations@marriott.com&quot; }  ).count()
    5

I can&#39;t see what I am doing wrong. I am expecting nested object notation to return the same result as the dot notation query. Where am I wrong?

How to query nested objects?

What is the command to show the current db in the MongoDB shell?

(I failed to find it on Google)

Command to show the db that I&#39;m currently using in MongoDB?

If I had a schema like this:

&lt;!-- language: lang-js --&gt;

    var person = new Schema({
      firstName:  String,
      lastName: String,
    });

I&#39;d like to ensure that there is only one document with the same firstName and lastName.

How can I accomplish this?




Mongoose: how to define a combination of fields to be unique?

Looking to do the following query:

    Entrant
        .find
          enterDate : oneMonthAgo
          confirmed : true
        .where(&#39;pincode.length &gt; 0&#39;)
        .exec (err,entrants)-&gt;

Am I doing the where clause properly? I want to select documents where `pincode` is not null.

Mongoose query where value is not null

I have a Mongoose schema with an array `lists` of objects that consist of a reference to another collection and a nested array of numbers:

    var Schema, exports, mongoose, schema;
    
    mongoose = require(&quot;mongoose&quot;);
    
    Schema = mongoose.Schema;
    
    schema = new Schema({
      name: {
        type: String,
        required: true,
        unique: true,
        trim: true
      },
      lists: [
        {
          list: {
            type: Schema.ObjectId,
            require: true,
            ref: &quot;List&quot;
          },
          allocations: [
            {
              type: Number,
              required: true
            }
          ]
        }
      ],
      createdAt: {
        type: Date,
        &quot;default&quot;: Date.now
      },
      updatedAt: {
        type: Date
      }
    });
    
    exports = module.exports = mongoose.model(&quot;Portfolio&quot;, schema);

However, I cannot get `populate` to work as expected without getting a `TypeError: Cannot read property &#39;ref&#39; of undefined`. I&#39;ve tried `populate(&#39;list&#39;)` and `populate(&#39;lists list&#39;)` but I&#39;m either not calling things correctly or my Schema isn&#39;t formed correctly. I don&#39;t have this problem if I simply reference the lists by themselves:

    lists: [
        {
            type: Schema.ObjectId,
            require: true,
            ref: &quot;List&quot;
        }
      ]

but I want to have the allocations array alongside each list. What do I need to do to get the behavior I want?

Mongoose populate with array of objects containing ref

This is my appointment collection:


        
    { _id: ObjectId(&quot;518ee0bc9be1909012000002&quot;), date: ISODate(&quot;2013-05-13T22:00:00Z&quot;), patient:ObjectId(&quot;518ee0bc9be1909012000002&quot;) }
    
    { _id: ObjectId(&quot;518ee0bc9be1909012000002&quot;), date: ISODate(&quot;2013-05-13T22:00:00Z&quot;), patient:ObjectId(&quot;518ee0bc9be1909012000002&quot;) }
    
    { _id: ObjectId(&quot;518ee0bc9be1909012000002&quot;), date: ISODate(&quot;2013-05-13T22:00:00Z&quot;), patient:ObjectId(&quot;518ee0bc9be1909012000002&quot;) }

I used aggregate to get the following result
        
    {date: ISODate(&quot;2013-05-13T22:00:00Z&quot;),
    patients:[ObjectId(&quot;518ee0bc9be1909012000002&quot;),ObjectId(&quot;518ee0bc9be1909012000002&quot;),ObjectId(&quot;518ee0bc9be1909012000002&quot;)] }

like this:


    Appointments.aggregate([
    {$group: {_id: &#39;$date&#39;, patients: {$push: &#39;$patient&#39;}}},
    {$project: {date: &#39;$_id&#39;, patients: 1, _id: 0}}
    ], ...)


How can I populate the patient document
I trued this but it doesn&#39;t work ... `Appointments.find({}).populate(&quot;patient&quot;).aggregate`....

In other words, can i use populate and aggregate at the same statement

any help please

how to use populate and aggregate in same statement?

Original question
----------------------

### Background

It is well-known that SQLite [needs to be fine tuned][1] to achieve insert speeds on the order of 50k inserts/s. There are many questions here regarding slow insert speeds and a wealth of advice and benchmarks.

There are also [claims that SQLite can handle large amounts of data][2], with reports of 50+ GB not causing any problems with the right settings.

I have followed the advice here and elsewhere to achieve these speeds and I&#39;m happy with 35k-45k inserts/s. The problem I have is that all of the benchmarks only demonstrate fast insert speeds with &lt; 1m records. What I am seeing is that _insert speed seems to be inversely proportional to table size_.

### Issue

My use case requires storing 500m to 1b tuples (`[x_id, y_id, z_id]`) over a few years (1m rows / day) in a link table. The values are all integer IDs between 1 and 2,000,000. There is a single index on `z_id`.

Performance is great for the first 10m rows, ~35k inserts/s, but by the time the table has ~20m rows, performance starts to suffer. I&#39;m now seeing about 100 inserts/s.

The size of the table is not particularly large. With 20m rows, the size on disk is around 500MB.

The project is written in Perl.

### Question

Is this the reality of large tables in SQLite or are there any secrets to _maintaining_ high insert rates for tables with &gt; 10m rows?

### Known workarounds which I&#39;d like to avoid if possible

* _Drop the index, add the records, and re-index_: This is fine as a workaround, but doesn&#39;t work when the DB still needs to be usable during updates. It won&#39;t work to make the database completely inaccessible for _x_ minutes / day
* _Break the table into smaller subtables / files_: This will work in the short term and I have already experimented with it. The problem is that I need to be able to retrieve data from the entire history when querying which means that eventually I&#39;ll hit the 62 table attachment limit. Attaching, collecting results in a temp table, and detaching hundreds of times per request seems to be a lot of work and overhead, but I&#39;ll try it if there are no other alternatives.
* _Set_ [`SQLITE_FCNTL_CHUNK_SIZE`][3]: I don&#39;t know C (?!), so I&#39;d prefer to not learn it just to get this done. I can&#39;t see any way to set this parameter using Perl though.


UPDATE
----------------

Following [Tim&#39;s suggestion][7] that an index was causing increasingly
slow insert times despite SQLite&#39;s claims that it is capable
of handling large data sets, I performed a benchmark comparison with the following 
settings:

* inserted rows: _14 million_
* commit batch size: _50,000 records_
* `cache_size` pragma: _10,000_
* `page_size` pragma: _4,096_
* `temp_store` pragma: _memory_
* `journal_mode` pragma: _delete_
* `synchronous` pragma: _off_

In my project, as in the benchmark results below, a file-based temporary table is created and SQLite&#39;s built-in support
for importing CSV data is used. The temporary table is then attached
to the receiving database and sets of 50,000 rows are inserted with an
`insert-select` statement. Therefore, the insert times do not reflect
_file to database_ insert times, but rather _table to table_ insert
speed. Taking the CSV import time into account would reduce the speeds
by 25-50% (a very rough estimate, it doesn&#39;t take long to import the
CSV data).

Clearly having an index causes the slowdown in insert speed as table size increases.

![Plot of SQLite insert speed and table size][4]

It&#39;s quite clear from the data above that the correct answer can be assigned to [Tim&#39;s answer][6] rather than the assertions that SQLite just can&#39;t handle it. Clearly it _can_ handle large datasets _if_ indexing that dataset is not part of your use case. I have been using SQLite for just that, as a backend for a logging system, for a while now which does _not_ need to be indexed, so I was quite surprised at the slowdown I experienced.


Conclusion
----------------------

If anyone finds themselves wanting to store a large amount of data using SQLite _and_ have it indexed, [using shards][5] may be the answer. I eventually settled on using the first three characters of an MD5 hash a unique column in `z` to determine assignment to one of 4,096 databases. Since my use case is primarily archival in nature, the schema will not change and queries will never require shard walking. There is a limit to database size since extremely old data will be reduced and eventually discarded, so this combination of sharding, pragma settings, and even some *de*normalisation gives me a nice balance that will, based on the benchmarking above, maintain an insert speed of at least 10k inserts / second.

  [Tim]: https://stackoverflow.com/users/311343/tim

  [1]: https://stackoverflow.com/questions/1711631/how-do-i-improve-the-performance-of-sqlite
  [2]: https://stackoverflow.com/questions/784173/what-are-the-performance-characteristics-of-sqlite-with-very-large-database-file?lq=1
  [3]: http://www.sqlite.org/c3ref/c_fcntl_chunk_size.html#sqlitefcntlchunksize
  [4]: http://i.stack.imgur.com/UMDAM.png
  [5]: https://stackoverflow.com/questions/128919/extreme-sharding-one-sqlite-database-per-user
  [6]: https://stackoverflow.com/a/15809806/918938
  [7]: https://stackoverflow.com/questions/15778716/sqlite-insert-speed-slows-as-number-of-records-increases-due-to-an-index#comment22443079_15778716

SQLite insert speed slows as number of records increases due to an index

I just created a Instead After Trigger whose syntax is given below:

    Create trigger tgrInsteadTrigger on copytableto
    Instead of Insert as 
        Declare @store_name varchar(30);
        declare @sales int;
        declare @date datetime;
    
        select @store_name = i.store_name from inserted i
        select @sales = i.sales from inserted i
        select @date = i.Date from inserted i
    begin
        if (@sales &gt; 1000)
            begin
            RAISERROR(&#39;Cannot Insert where salary &gt; 1000&#39;,16,1); ROLLBACK;
            end
        else
            begin
            insert into copytablefrom(store_name, sales, date) values (@store_name, @sales, @date);
            Print &#39;Instead After Trigger Executed&#39;;
            end
    End

In the above syntax I have used `RAISERROR(&#39;Cannot Insert where salary &gt; 1000&#39;,16,1)`

But when I write `RAISERROR(&#39;Cannot Insert where salary &gt; 1000&#39;)` it gives the error &quot;Incorrect syntax near &#39;)&#39;&quot; on the same line.

Can anyone please explain the use of (16,1) here.

What is the syntax meaning of RAISERROR()

*Preface: I&#39;m attempting to use the repository pattern in an MVC architecture with relational databases.*

I&#39;ve recently started learning TDD in PHP, and I&#39;m realizing that my database is coupled much too closely with the rest of my application. I&#39;ve read about repositories and using an [IoC container][1] to &quot;inject&quot; it into my controllers. Very cool stuff. But now have some practical questions about repository design. Consider the follow example.

&lt;!-- language: lang-php --&gt;

    &lt;?php

    class DbUserRepository implements UserRepositoryInterface
    {
        protected $db;

        public function __construct($db)
        {
            $this-&gt;db = $db;
        }

        public function findAll()
        {
        }

        public function findById($id)
        {
        }

        public function findByName($name)
        {
        }

        public function create($user)
        {
        }

        public function remove($user)
        {
        }

        public function update($user)
        {
        }
    }


## Issue #1: Too many fields

All of these find methods use a select all fields (`SELECT *`) approach. However, in my apps, I&#39;m always trying to limit the number of fields I get, as this often adds overhead and slows things down. For those using this pattern, how do you deal with this?

## Issue #2: Too many methods

While this class looks nice right now, I know that in a real-world app I need a lot more methods. For example:

- findAllByNameAndStatus
- findAllInCountry
- findAllWithEmailAddressSet
- findAllByAgeAndGender
- findAllByAgeAndGenderOrderByAge
- Etc.

As you can see, there could be a very, very long list of possible methods. And then if you add in the field selection issue above, the problem worsens. In the past I&#39;d normally just put all this logic right in my controller:

&lt;!-- language: lang-php --&gt;

    &lt;?php
    
    class MyController
    {
        public function users()
        {
            $users = User::select(&#39;name, email, status&#39;)
                -&gt;byCountry(&#39;Canada&#39;)-&gt;orderBy(&#39;name&#39;)-&gt;rows();
    
            return View::make(&#39;users&#39;, array(&#39;users&#39; =&gt; $users));
        }
    }

With my repository approach, I don&#39;t want to end up with this:

&lt;!-- language: lang-php --&gt;

    &lt;?php

    class MyController
    {
        public function users()
        {
            $users = $this-&gt;repo-&gt;get_first_name_last_name_email_username_status_by_country_order_by_name(&#39;Canada&#39;);

            return View::make(&#39;users&#39;, array(&#39;users&#39; =&gt; $users))
        }

    }

## Issue #3: Impossible to match an interface

I see the benefit in using interfaces for repositories, so I can swap out my implementation (for testing purposes or other). My understanding of interfaces is that they define a contract that an implementation must follow. This is great until you start adding additional methods to your repositories like `findAllInCountry()`. Now I need to update my interface to also have this method, otherwise, other implementations may not have it, and that could break my application. By this feels insane...a case of the tail wagging the dog.

### Specification Pattern?
This leads me to believe that repository should only have a fixed number of methods (like `save()`, `remove()`, `find()`, `findAll()`, etc). But then how do I run specific lookups? I&#39;ve heard of the [Specification Pattern][2], but it seems to me that this only reduces an entire set of records (via `IsSatisfiedBy()`), which clearly has major performance issues if you&#39;re pulling from a database.

## Help?

Clearly, I need to rethink things a little when working with repositories. Can anyone enlighten on how this is best handled?

  [1]: http://laravel.com/docs/4.2/ioc
  [2]: http://en.wikipedia.org/wiki/Specification_pattern

Proper Repository Pattern Design in PHP?

I have a ton of postgresql dump files I need to peruse through for data. Do I have to install Postgresql and &quot;recover&quot; each one of them into new databases one by one? Or I&#39;m hoping there&#39;s a postgresql client that can simply open them up and I can peek at the data, maybe even run a simple SQL query?

The dump files are all from a Postgresql v9.1.9 server.

Or maybe there&#39;s a tool that can easily make a database &quot;connection&quot; to the dump files?

***UPDATE***: These are **not text files**. They are **binary**. They come from Heroku&#39;s backup mechanism, this is [what Heroku says about how they create their backups][1]:

&gt; PG Backups uses the native pg_dump PostgreSQL tool to create its
&gt; backup files, making it trivial to export to other PostgreSQL
&gt; installations.


  [1]: https://devcenter.heroku.com/articles/heroku-postgres-import-export

Easy way to view postgresql dump files?

I&#39;m in the process of moving my files onto another computer, and one thing I would like to do is to transfer the data in my MySQL database. I want to dump the databases into .sql files, but also have the create database db_name including in the file, that way I just have to import the file to mysql without having to manually create the databases. Is there a way to do that?

Content Type	Original Author	Original Content on Stackoverflow
Question	Ari Porad	View Question on Stackoverflow
Solution 1 - Javascript	jmar777	View Answer on Stackoverflow

What is the difference between id and _id in mongoose?

Javascript Problem Overview

Javascript Solutions

Solution 1 - Javascript

How do I ensure that whitespace is preserved in Markdown?

How do you print a C++11 time_point?

Attributions