I would like to do the equivalent of this SQL but with Solr as my data store.

    SELECT
       DISTINCT txt
    FROM
       my_table;

What syntax would force Solr to only give me distinct values?

    http://localhost:8983/solr/select?q=txt:?????&amp;fl=txt

**EDIT:** So faceted searching seems to fit, but as I investigated it, I realized I had only detailed half of the problem.

My SQL query should have read...

    SELECT
       DISTINCT SUBSTR(txt,0,3)
    FROM
       my_table;

Any possibility of this with Solr?

How to select distinct field values using Solr?

My team is working with a third party CMS that uses Solr as a search index.  I&#39;ve noticed that it seems like the authors are using Solr as a database of sorts in that each document returned contains two fields:

 1. The Solr document ID (basically a classname and database id)
 2. An XML representation of the entire object

So basically it runs a search against Solr, download the XML representation of the object, and then instantiate the object from the XML rather than looking it up in the database using the id.

My gut feeling tells me this is a bad practice.  Solr is a search index, not a database... so it makes more sense to me to execute our complex searches against Solr, get the document ids, and then pull the corresponding rows out of the database.

Is the current implementation perfectly sound, or is there data to support the idea that this is ripe for refactoring?

**EDIT:** When I say &quot;XML representation&quot; - I mean one stored field that contains an XML string of all of the object&#39;s properties, not multiple stored fields.

Using Solr search index as a database - is this &quot;wrong&quot;?


I struggle with understanding the difference between **collections and cores**. If I understand it correctly, cores are multiple indexes. Collection consists of cores, so essentially they share the same logic in separation, i.e. separate cores and collections have separate end-points.

I have the following scenario. I create a backend for cloud service for several online shops. Each shop has a set of products, to which customers can add reviews. I want to index static data (product information) separately from dynamic information(reviews) so I can improve performance.

How can I best separate in Solr???

Solr Collection vs Cores

[SOLR docs](http://wiki.apache.org/solr/CommonQueryParameters#fq), state that **filter queries**, unlike the **main query**, do not influence the document score. Can anyone explain what does this mean exactly, preferably with an example.

Thanks.

SOLR filter-query vs main-query

We need to integrate a search engine in our Product Catalog management software. the catalog is expected to have more than 4-5 mn. records with relational data spread over several tables. Our dev platform is Asp.Net 3.5 and we have done some pre-liminary work on Lucene, found it to be good. However, we just came to know of Solr and was looking for some practical tips to compare Lucene &amp; Solr from implementation, timeline, regular maintenance, performance, features perspective. Any guidance or pointers would be really helpful. Thanks. 

Search Engine - Lucene or Solr

At the moment I have to go to `/usr/java/apache-solr-1.4.0/example` and then do:

    java -jar start.jar

How do I get this to start automatically on boot? 

I&#39;m on a shared Linux server.


How to start Solr automatically?

With the NoSQL movement growing based on document-based databases, I&#39;ve looked at MongoDB lately.  I have noticed a striking similarity with how to treat items as &quot;Documents&quot;, just like Lucene does (and users of Solr).

So, the question: **Why would you want to use NoSQL (MongoDB, Cassandra, CouchDB, etc) over Lucene (or Solr) as your &quot;database&quot;?**

What I am (and I am sure others are) looking for in an answer is some deep-dive comparisons of them.  Let&#39;s skip over relational database discussions all together, as they serve a different purpose. 

Lucene gives some serious advantages, such as powerful searching and weight systems.  Not to mention facets in Solr (which Solr is being integrated into Lucene soon, yay!).  You can use Lucene documents to store IDs, and access the documents as such just like MongoDB.  Mix it with Solr, and you now get a WebService-based, load balanced solution.

You can even throw in a comparison of out-of-proc cache providers such as Velocity or MemCached when talking about similar data storing and scalability of MongoDB.

The restrictions around MongoDB reminds me of using MemCached, but I can use Microsoft&#39;s Velocity and have more grouping and list collection power over MongoDB (I think).  Can&#39;t get any faster or scalable than caching data in memory.  Even Lucene has a memory provider.
 
MongoDB (and others) do have some advantages, such as the ease of use of their API.  New up a document, create an id, and store it.  Done.  Nice and easy.



NoSQL (MongoDB) vs Lucene (or Solr) as your database

I&#39;m looking for a stand-alone full-text search server with the following properties:

* Must operate as a stand-alone server that can serve search requests from multiple clients
* Must be able to do &quot;bulk indexing&quot; by indexing the result of an SQL query: say &quot;SELECT id, text_to_index FROM documents;&quot;
* Must be free software and must run on Linux with MySQL as the database
* Must be fast (rules out MySQL&#39;s internal full-text search)

The alternatives I&#39;ve found that have these properties are:

* Solr (based on Lucene)
* ElasticSearch (also based on Lucene)
* Sphinx

My questions:

* How do they compare? 
* Have I missed any alternatives?
* I know that each use case is different, but are there certain cases where I would definitely *not* want to use a certain package?


Choosing a stand-alone full-text search server: Sphinx or SOLR?

I know that Lucene and Solr are 2 differents [Apache projects][1] that are made to work together, but I don&#39;t understand what is the aim of each project.

For what I understood until now is that Lucene is used to create a search index and Solr use this index to perform searches.
Am I right or is this a totally different approach?


  [1]: http://lucene.apache.org/

Difference between solr and lucene

What are the core architectural differences between these technologies?  

Also, what use cases are generally more appropriate for each?



Solr vs. ElasticSearch

I have a large solr index, and I have noticed some fields are not updated correctly (the index is dynamic).

This has resulted in some fields having an empty &quot;id&quot; field.

I have tried these queries, but they didn&#39;t work:

     id:&#39;&#39;
     id:NULL
     id:null
     id:&quot;&quot;
     id:
     id:[&#39;&#39; TO *]

Is there a way to query empty fields?

Thanks

How to query SOLR for empty fields?

How do I delete all data from `solr` by command?  We are using `solr` with `lily` and `hbase`.

How can I delete data from both hbase and solr?

http://lucene.apache.org/solr/4_10_0/tutorial.html#Deleting+Data


How to delete all data from solr and hbase

I&#39;m currently looking at other search methods rather than having a huge SQL query.
I saw [elasticsearch][1] recently and played with [whoosh][2] (a Python implementation of a search engine).

Can you give reasons for your choice(s)?


  [1]: http://elasticsearch.org/
  [2]: http://whoosh.ca/ &quot;Whoosh&quot;

ElasticSearch, Sphinx, Lucene, Solr, Xapian. Which fits for which usage?

I&#39;m working on a solr query similar to the following:

    ((myField:superneat AND myOtherField:somethingElse) OR NOT myField:superneat)

When running this, no results are returned.  Using criteria on either side of the OR NOT returns results that I&#39;d expect - they are just not working well together.  In the case that **myField** matches *superneat*, I&#39;m intending to also ensure that **myOtherField** is set to *somethingElse*, but if **myField** is not *superneat*, include it in the results.

Can someone explain why solr is not returning results for this kind of query?  Should the query be restructured somehow - or is there a different way in which solr can be used to achieve the desired result?

using OR and NOT in solr query

I am a little confused as to what the behaviour of the index and stored attibutes of the Solr fields is.

For example if I have the following in the Schema.xml 

    &lt;field name=&quot;test1&quot; type=&quot;text&quot; indexed=&quot;false&quot;
            stored=&quot;false&quot; required=&quot;false&quot; /&gt;

Will the field **test1** be not stored in the Solr document even if I create a document with that field in it and set a value to that field and commit the document to Solr. As I have the `stored=false` attribute, does it mean that the value of the field is lost in Solr and not persisted?

Solr index vs stored

I&#39;m new to Apache Solr.  Even after reading the [documentation][1] part, I&#39;m finding it difficult to clearly understand the functionality and use of the `multiValued` field type property. 

What internally Solr does/treats/handles a field that is marked as  *multiValued*? 

What is the difference in indexing in Solr between a field that is *multiValued* and those that are not?

Can somebody explain with some good example?

Doc says:

&gt; *multiValued=true|false*

&gt; True if this
&gt; field may contain multiple values per
&gt; document, i.e. if it can appear
&gt; multiple times in a document

  [1]: http://wiki.apache.org/solr/SchemaXml &quot;Solr Schema XML&quot;

What is the use of &quot;multiValued&quot; field type in Solr?

Please tell me how can I determine the version number of currently running solr ?


How do I find out version of currently running Solr?

How do I delete all the documents in my SOLR index using the SOLR Admin.

I tried using the url and it works but want to know if the same can be done using the Admin..



Deleting solr documents from Solr Admin

I&#39;m preparing to deploy a Rails app on Heroku that requires full text search. Up to now I&#39;ve been running it on a VPS using MySQL with Sphinx.

However, if I want to use Sphinx or Solr on Heroku, I&#39;d need to pay for an add-on.

I notice that PostgreSQL (the DB used on Heroku) has built-in full text search capability.

Is there a reason I couldn&#39;t use Postgres&#39;s full-text search? Is it slower than Sphinx or is there some other major limitation?

Any reason not use PostgreSQL&#39;s built-in full text search on Heroku?

I recently switched from Postgres to Solr and saw a ~50x speed up in our queries. The queries we run involve multiple ranges, and our data is vehicle listings. For example: &quot;Find all vehicles with mileage &lt; 50,000, $5,000 &lt; price &lt; $10,000, make=Mazda...&quot;

I created indices on all the relevant columns in Postgres, so it should be a pretty fair comparison. Looking at the query plan in Postgres though it was still just using a single index and then scanning (I assume because it couldn&#39;t make use of all the different indices).

As I understand it, Postgres and Solr use vaguely similar data structures (B-trees), and they both cache data in-memory. So I&#39;m wondering where such a large performance difference comes from.

What differences in architecture would explain this?

Why is Solr so much faster than Postgres?

I have a classifieds website. Users may put ads, edit ads, view ads etc.

Whenever a user puts an ad, I am adding a document to Solr.
I don&#39;t know, however, when to commit it. Commit slows things down from what I have read.

How should I do it? Autocommit every 12 hours or so?

Also, how should I do it with optimize?

Solr commit and optimize questions

All of us have come across the recent hype of no-SQL solutions lately. MongoDB, CouchDB, BigTable, Cassandra, and others have been listed as no-SQL options. Here&#39;s an example:

http://architects.dzone.com/articles/what-nosql-store-should-i-use

However, three years ago a co-worker and I were using Lucene.NET as what seem to fit the description of no-SQL. We did not use it just for user-inputted search queries; we used it to make a few reindexed RDBMS table data extremely performant. We implemented our own .NET sort-of-equivalent-to-Solr service to manage these indexes and make them callable. When I left the company, the team switched to Solr itself. (For those not in the know, Solr is a web service that wraps Lucene with REST-callable queries and index dumps.)

What I don&#39;t understand is, why is Solr not counted in the typical lists of no-SQL solution options? Am I missing something here? I assume that there are technical reasons why Solr is not comparable to the likes of CouchDB, etc., and in fact I understand that CouchDB uses Lucene as its data store (yes?), but what disqualifies Solr? 

I&#39;m not asking as some kind of Solr fanboy or anything, I just don&#39;t understand why Solr and the like don&#39;t fit the definition of no-SQL, and if Solr technically does fit the definition then what about it likely makes people pooh-pooh it? I&#39;m asking because I&#39;m having difficulty determining whether I should continue using Lucene-based solutions (like Solr) for solutions that I build or if I should really do more research with these other options.

Why are document stores like Lucene / Solr not included in NoSQL conversations?

In apache Solr why do we always need to prefer string field over text field if both solves purposes?

How string or text affects the parameters like index size, index read, index creation? 

Apache Solr string field or text field?

I am running a Solr instance on Jetty and when I search using the Solr admin panel, it returns the entire document. What should I do to get only specified fields from each Solr document returned by the search?

All Solr Solutions on Gang of Coders

Total of 24 Solr Solutions