How to create only one table with SQLAlchemy?

PythonMysqlPython 3.xSqlalchemy

Python Problem Overview


I am unable to create a single table using SQLAlchemy.

I can create it by calling Base.metadata.create_all(engine) but as the number of table grows, this call takes a long time.

I create table classes on the fly and then populate them.

from sqlalchemy import create_engine, Column, Integer, Sequence, String, Date, Float, BIGINT
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

Base = declarative_base()

class HistoricDay():
	
	id = Column(Integer, Sequence('id_seq'), primary_key=True)
	#  Date, Open, High, Low, Close, Volume, Adj Close
	date = Column(Date)
	open = Column(Float)
	high = Column(Float)
	low = Column(Float)
	close = Column(Float)
	volume = Column(BIGINT)
	adjClose = Column(Float)
	
	def __init__(self, date, open, high, low, close, volume, adjClose):
		self.date = date
		self.open = open
		self.high = high
		self.low = low
		self.close = close
		self.volume = volume
		self.adjClose = adjClose

def build_daily_history_table_repr(self):
		return "<"+self.__tablename__+"('{}','{}','{}','{}','{}','{}','{}','{}')>".format(self.id, self.date, self.open, self.high, self.low, self.close, self.volume, self.adjClose)
		
def build_daily_history_table(ticket):
	classname = ticket+"_HistoricDay"
	globals()[classname] = type(classname, (HistoricDay,Base), {'__tablename__' : ticket+"_daily_history"})
	setattr(globals()[classname], '__repr__', build_daily_history_table_repr)

# Initialize the database :: Connection & Metadata retrieval
engine = create_engine('mysql+cymysql://root@localhost/gwc?charset=utf8&use_unicode=0', pool_recycle=3600) # ,echo = True

# SqlAlchemy :: Session setup
Session = sessionmaker(bind=engine)

# Create all tables that do not already exist
Base.metadata.create_all(engine)

# SqlAlchemy :: Starts a session
session = Session()

ticketList = getTicketList()

for ticket in ticketList:
	build_daily_history_table(ticket)
	class_name = ticket+"_HistoricDay"
	
	meta_create_all_timer = time.time()
	# Create all tables that do not already exist
	# globals()[class_name]('2005-07-24',0,0,0,0,0,0).create(engine)  #doesn't work
	#(globals()[class_name]).__table__.create(engine) #doesn't work
	# session.commit() #doesn't work
	
	#Base.metadata.create_all(engine) # works but gets very slow
	print("  meta_create_all_timer {}s".format(time.time()-meta_create_all_timer))
	
    data = getData(ticket)

	for m_date, m_open, m_close, m_high, m_low, m_volume, m_adjClose in data:
		entry = globals()[class_name](m_date, m_open, m_high, m_low, m_close, m_volume, m_adjClose)
		session.add(entry)
	
	session.commit()

I saw in the documentation that you can do

engine = create_engine('sqlite:///:memory:')

meta = MetaData()

employees = Table('employees', meta,
    Column('employee_id', Integer, primary_key=True),
    Column('employee_name', String(60), nullable=False, key='name'),
    Column('employee_dept', Integer, ForeignKey("departments.department_id"))
)
employees.create(engine)

However, I'm not able to figure out how to do the same thing as Table does, with declarative_base().

How can I do that with classes that inherit from declarative_base()?

Python Solutions


Solution 1 - Python

> Above, the declarative_base() callable returns a new base class from > which all mapped classes should inherit. When the class definition is > completed, a new Table and mapper() will have been generated. > > The resulting table and mapper are accessible via __table__ and > __mapper__ attributes

(From here)

Therefore:

def build_daily_history_table(ticket):
    classname = ticket + "_HistoricDay"
    ticket = type(classname, (Base, HistoricDay), {'__tablename__' : ticket+"_daily_history"})
    ticket.__repr__ =  build_daily_history_table_repr
    return ticket

build_daily_history_table("test").__table__.create(bind = engine)

Output:

2013-10-04 22:36:53,263 INFO sqlalchemy.engine.base.Engine 
CREATE TABLE test_daily_history (
	id INTEGER NOT NULL, 
	date DATE, 
	open FLOAT, 
	high FLOAT, 
	low FLOAT, 
	close FLOAT, 
	volume BIGINT, 
	"adjClose" FLOAT, 
	PRIMARY KEY (id)
)


2013-10-04 22:36:53,263 INFO sqlalchemy.engine.base.Engine ()
2013-10-04 22:36:53,263 INFO sqlalchemy.engine.base.Engine COMMIT

Credit goes to javex's comment/correction or I might have suggested something akin to:

Base.metadata.tables["ticket_daily_history"].create(bind = engine)

Advise:

The approach used in build_daily_history_table could be one of the least elegant ways of doing things, primarily for the reason that it is polluting/cluttering the namespace.

Solution 2 - Python

To create specific tables, giving tables parameter to create_all() method is enough.

Base.metadata.create_all(engine, tables=table_objects)

table_objects equals to:

table_objects = [HistoricDay.__table__]

or

table_objects = [Base.metadata.tables["historicday"]]

I showed one table here. You can increase the number of the tables as you wish.

Reference: http://docs.sqlalchemy.org/en/latest/core/metadata.html#sqlalchemy.schema.MetaData.create_all

Solution 3 - Python

Create all not exists tables with one line. It will check whether the table exists first by default.

Base.metadata.create_all(db_engine, Base.metadata.tables.values(),checkfirst=True)

Create one target table with table_name.

Base.metadata.create_all(db_engine, Base.metadata.tables[table_name],checkfirst=True)

It works perfectly with declarative_base.

from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()
TABLE_PREFIX = "decision_"


class Stock(Base):
	__tablename__ = '{}stocks'.format(TABLE_PREFIX)

	id = Column(Integer, primary_key=True)
	name = Column(String)

class StagePerformance(Base):
	__tablename__ = '{}stage_performance'.format(TABLE_PREFIX)

	id = Column(Integer, primary_key=True)
	date = Column(DateTime)
	stock = relationship("Stock", back_populates="stage_performances")
	period = Column(Integer )
	open = Column(Float)
	high = Column(Float)
	low = Column(Float)
	close = Column(Float)
	change_ratio = Column(Float)
	turnover = Column(Float)
	volume = Column(Float)

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionLazikView Question on Stackoverflow
Solution 1 - PythonBleeding FingersView Answer on Stackoverflow
Solution 2 - PythonvlyalcinView Answer on Stackoverflow
Solution 3 - PythonW.PerrinView Answer on Stackoverflow