Postgres: Distinct but only for one column

PostgresqlSelectDistinct

Postgresql Problem Overview


I have a table on pgsql with names (having more than 1 mio. rows), but I have also many duplicates. I select 3 fields: id, name, metadata.

I want to select them randomly with ORDER BY RANDOM() and LIMIT 1000, so I do this is many steps to save some memory in my PHP script.

But how can I do that so it only gives me a list having no duplicates in names.

For example [1,"Michael Fox","2003-03-03,34,M,4545"] will be returned but not [2,"Michael Fox","1989-02-23,M,5633"]. The name field is the most important and must be unique in the list everytime I do the select and it must be random.

I tried with GROUP BY name, bu then it expects me to have id and metadata in the GROUP BY as well or in a aggragate function, but I dont want to have them somehow filtered.

Anyone knows how to fetch many columns but do only a distinct on one column?

Postgresql Solutions


Solution 1 - Postgresql

To do a distinct on only one (or n) column(s):

select distinct on (name)
    name, col1, col2
from names

This will return any of the rows containing the name. If you want to control which of the rows will be returned you need to order:

select distinct on (name)
    name, col1, col2
from names
order by name, col1

Will return the first row when ordered by col1.

distinct on:

> SELECT DISTINCT ON ( expression [, ...] ) keeps only the first row of each set of rows where the given expressions evaluate to equal. The DISTINCT ON expressions are interpreted using the same rules as for ORDER BY (see above). Note that the “first row” of each set is unpredictable unless ORDER BY is used to ensure that the desired row appears first. > >The DISTINCT ON expression(s) must match the leftmost ORDER BY expression(s). The ORDER BY clause will normally contain additional expression(s) that determine the desired precedence of rows within each DISTINCT ON group.

Solution 2 - Postgresql

> Anyone knows how to fetch many columns but do only a distinct on one column?

You want the DISTINCT ON clause.

You didn't provide sample data or a complete query so I don't have anything to show you. You want to write something like:

SELECT DISTINCT ON (name) fields, id, name, metadata FROM the_table;

This will return an unpredictable (but not "random") set of rows. If you want to make it predictable add an ORDER BY per Clodaldo's answer. If you want to make it truly random, you'll want to ORDER BY random().

Solution 3 - Postgresql

To do a distinct on n columns:

select distinct on (col1, col2) col1, col2, col3, col4 from names

Solution 4 - Postgresql

SELECT NAME,MAX(ID) as ID,MAX(METADATA) as METADATA 
from SOMETABLE
GROUP BY NAME

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionNovumCoderView Question on Stackoverflow
Solution 1 - PostgresqlClodoaldo NetoView Answer on Stackoverflow
Solution 2 - PostgresqlCraig RingerView Answer on Stackoverflow
Solution 3 - PostgresqlSunil KumarView Answer on Stackoverflow
Solution 4 - PostgresqlDavid JashiView Answer on Stackoverflow