Makefile issue: smart way to scan directory tree for .c files

I am doing a project which is growing pretty fast and keeping the object files up date is no option. The problem beyond wildcard command lies somewhere between "I do not want recursive makefiles" and "I do not want it to list by hand". The objects are supposed to go into a separate directory, which works already. Note: I am not that used to makefiles, I know the basics, but everything beyond...

So my question: How to scan a src folder recursively and do that in a smart manner?

I already did it with multiple SRC variables but that's ugly and clutters the whole makefile with an increasing number of directories.

What I currently use is:

OS = Linux

VERSION = 0.0.1
CC      = /usr/bin/gcc
CFLAGS  = -Wall -g -D_REENTRANT -DVERSION=\"$(VERSION)\"
LDFLAGS = -lm `pkg-config --cflags gtk+-2.0` `pkg-config --libs gtk+-2.0`

BUILDDIR = build
SOURCEDIR = src
HEADERDIR = src

SOURCES = $(wildcard $(SOURCEDIR)/*.c)
OBJECTS = $(patsubst $(SOURCEDIR)/%.c, $(BUILDDIR)/%.o, $(SOURCES))

NAME = cinnamon
BINARY = cinnamon.bin

ECHO = echo
RM = rm -rf
MKDIR = mkdir
INSTALL = install

.PHONY: all clean setup

all: $(BINARY)


$(BINARY): $(BUILDDIR)/$(OBJECTS)
	$(CC) $(CFLAGS) $(LDFLAGS) -I$(HEADERDIR) -I$(SOURCEDIR) $(OBJECTS) -o $(BINARY) 


$(BUILDDIR)/%.o: $(SOURCEDIR)/%.c
	$(CC) $(CFLAGS) $(LDFLAGS) -I$(HEADERDIR) -I$(SOURCEDIR) -c $< -o $@

setup:
	$(MKDIR) -p $(BUILDDIR)

install:
	$(INSTALL) -m 755 -o 0 -g 0 -d $(DESTDIR)/usr/local/bin/
	$(INSTALL) -m 755 -o 0 -g 0 $(BINARY) $(DESTDIR)/usr/local/bin/$(BINARY)
	$(INSTALL) -m 755 -o 0 -g 0 -d $(DESTDIR)/usr/local/$(NAME)/ui/
	$(INSTALL) -m 644 -o 0 -g 0 ./ui/*.ui $(DESTDIR)/usr/local/$(NAME)/ui/
#	$(INSTALL) -m 755 -o 0 -g 0 -d $(DESTDIR)/usr/local/$(NAME)/model/
#	$(INSTALL) -m 644 -o 0 -g 0 ./model/*.model $(DESTDIR)/usr/local/$(NAME)/model/

clean:
	$(RM) $(BINARY) $(OBJECTS)

distclean: clean


help:
	@$(ECHO) "Targets:"
	@$(ECHO) "all     - buildcompile what is necessary"
	@$(ECHO) "clean   - cleanup old .o and .bin"
	@$(ECHO) "install - not yet fully supported"

Thanks to answer #1 it boils down to how to solve this:

$(BUILDDIR)/%.o: $(SOURCEDIR)/%.c
	$(CC) $(CFLAGS) $(LDFLAGS) $(SOURCETREE) -c $< -o $@

especially in the case of BUILDDIR = build and SOURCEDIR having to be replaced with the single .c files from SOURCES including their paths :/

The simplest option to do what you want is probably to just use a shell escape and call find:

SOURCES := $(shell find $(SOURCEDIR) -name '*.c')

This gets you a list of source files with paths. Note that the use of immediate assignment := rather than recursive assignment = is important here: you do not want to be running the shell escape every time SOURCES is inspected by make (which happens a lot more than you'd think in Makefiles). A general rule I find helpful is to always use immediate assignment unless I actually need recursive expansion (which is rare; it looks like all of your assignments in this example could be immediate). This then means use of recursive assignment is also a helpful signal that the variable needs to be used carefully.

Back to your problem. What you do next depends on whether you want a mirror of your source tree in your build tree, or whether the build dir is just supposed to contain a flat list of object files for all your source files, or whether you want a separate build dir under every source dir in the tree.

Assuming you want the mirrored build tree, you could do something like the following:

# Get list of object files, with paths
OBJECTS := $(addprefix $(BUILDDIR)/,$(SOURCES:%.c=%.o))

$(BINARY): $(OBJECTS)
    $(CC) $(CFLAGS) $(LDFLAGS) $(OBJECTS) -o $(BINARY)

$(BUILDDIR)/%.o: %.c
    $(CC) $(CFLAGS) $(LDFLAGS) -I$(HEADERDIR) -I$(dir $<) -c $< -o $@

This doesn't quite take into account the full complexity of the job, as it doesn't ensure the directories in the build tree actually exist (which would be moderately painful to do in Makefile syntax).

I removed the -I directives from your $(BINARY) build rule; do you really need them when linking objects? The reason I didn't leave them is that you don't have just one source dir anymore, and it's non-trivial to get the list of source dirs from the list of objects (like so much in Makefile syntax it would be doable but really annoying).

Recursive wildcards can be done purely in Make, without calling the shell or the find command. Doing the search using only Make means that this solution works on Windows as well, not just *nix.

# Make does not offer a recursive wildcard function, so here's one:
rwildcard=$(wildcard $1$2) $(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2))

# How to recursively find all files with the same name in a given folder
ALL_INDEX_HTMLS := $(call rwildcard,foo/,index.html)

# How to recursively find all files that match a pattern
ALL_HTMLS := $(call rwildcard,foo/,*.html)

The trailing slash in the folder name is required. This rwildcard function does not support multiple wildcards the way that Make's built-in wildcard function does, but adding that support would be straightforward with a couple more uses of foreach.

I like to do the following.

Create Variables to Each Directory of the Project

SRCDIR = src                                                           
OBJDIR = obj
LIBDIR = lib
DOCDIR = doc
HDRDIR = include

CFLAGS = -g -Wall -O3

Get Only the Internal Structure of SRCDIR Recursively

STRUCTURE := $(shell find $(SRCDIR) -type d)     

Get All Files inside the STRUCTURE Variable

CODEFILES := $(addsuffix /*,$(STRUCTURE))
CODEFILES := $(wildcard $(CODEFILES))            

Filter Out Only Specific Files

# Filter Only Specific Files                                
SRCFILES := $(filter %.c,$(CODEFILES))
HDRFILES := $(filter %.h,$(CODEFILES))
OBJFILES := $(subst $(SRCDIR),$(OBJDIR),$(SRCFILES:%.c=%.o))

# Filter Out Function main for Libraries
LIBDEPS := $(filter-out $(OBJDIR)/main.o,$(OBJFILES))

Now it is Time to create the Rules

compile: $(OBJFILES)

$(OBJDIR)/%.o: $(addprefix $(SRCDIR)/,%.c %.h)
    $(CC) -c $< -o $@ $(CFLAGS) 

With this approach, you can see that I'm using the STRUCTURE variable only to get the files inside the SRCDIR directory, but it can be used for others purposes as well, like mirror the SRCDIR inside OBJDIR once STRUCTURE stores only the internal sub-directories. It is quite useful after clean operations like:

clean:
    -rm -r $(OBJDIR)/*

NOTE: The compile rule only works well if for each *.c there is the corresponding *.h file (with the same base name, I mean).

Another good solution to this problem appears to be - implement a non-recursive makefile such as the one described here: http://sites.e-advies.nl/nonrecursive-make.html. This approach is nice because it seems fairly scalable - developers can add dependency information in the directory with the source files without having to worry about the overall makefile.