README.org 13 KB
Newer Older
martin.slouf's avatar
martin.slouf committed
1
# -*- mode: org -*-
2
# Time-stamp: <2021-09-15 20:47:36 martin>
martin.slouf's avatar
martin.slouf committed
3
4
5
6
7
8
9
#+TITLE: README.txt
#+AUTHOR: martinslouf@sourceforge.net
#+OPTIONS: ^:nil
#+LANGUAGE: en

* Description

martin.slouf's avatar
martin.slouf committed
10
11
12
13
Summer is light weight Python 3 application framework to support generic
application development.  It provides support for business object
management, ORM (mapping, declarative transactions), LDAP and localization.
Inspired by famous Java Spring application framework.
14

15
Since 0.5.2 Python 3 is required. Tested for latest Python 3.7 and Python 3.9.
martin.slouf's avatar
martin.slouf committed
16

17
18
19
Project web page https://gitlab.com/martin.slouf/py-summer.

Old project web page: http://py-summer.sourceforge.net/.
martin.slouf's avatar
martin.slouf committed
20

Martin Šlouf's avatar
Martin Šlouf committed
21

Martin Šlouf's avatar
Martin Šlouf committed
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
* Git Structure

Project follows this Git branching model:
https://nvie.com/posts/a-successful-git-branching-model/.

STARnet branches: [[./README-git.png]]

#+BEGIN_SRC dot :file README-git.png :cmdline -Tpng
# dot -Tpng <input.dot> -o <output.png>
digraph "STARnet git branches" {
        master [style="filled", label="master"];
        develop [style="filled", label="develop"];
        feature [label="feature/*"];
        hotfix [label="hotfix/*"];

        master -> {develop, hotfix, feature} [color="coral"];

        hotfix -> master [label="merge to (once ready)", color="gold", style="dashed"];
        feature -> develop [label="merge to (once ready)", color="gold", style="dashed"]
        feature -> master [label="backport", color="gold"]
        develop -> master [label="release", color="gold"]
}
#+END_SRC

martin.slouf's avatar
martin.slouf committed
46

martin.slouf's avatar
martin.slouf committed
47
* Installation
martin.slouf's avatar
martin.slouf committed
48

martin.slouf's avatar
martin.slouf committed
49
50
51
** Installing from PyPI (Python Package Index)

#+BEGIN_SRC sh
martin.slouf's avatar
martin.slouf committed
52
pip install summer
53
54
pip install summer[sqlalchemy] # with SQLAlchemy support
pip install summer[sqlalchemy,ldap] # with SQLAlchemy & LDAP support
martin.slouf's avatar
martin.slouf committed
55
56
#+END_SRC

martin.slouf's avatar
martin.slouf committed
57

martin.slouf's avatar
martin.slouf committed
58
59
60
** Installing from source distribution

Installation uses Python /setuptools/ -- just run from distribution top
martin.slouf's avatar
martin.slouf committed
61
62
dir:

martin.slouf's avatar
martin.slouf committed
63
#+BEGIN_SRC sh
martin.slouf's avatar
martin.slouf committed
64
65
66
67
68
python setup.py install
#+END_SRC

or you may prefer

martin.slouf's avatar
martin.slouf committed
69
#+BEGIN_SRC sh
martin.slouf's avatar
martin.slouf committed
70
71
72
73
74
75
76
python setup.py install --prefix=/usr/local/
#+END_SRC

This will copy the package into appropriate directory for third-party
modules of your Python installation.


martin.slouf's avatar
martin.slouf committed
77
78
79
80
** Setting up a development/runtime environment on Debian GNU/Linux

#+BEGIN_SRC sh
# install base python packages from your distribution repo
martin.slouf's avatar
martin.slouf committed
81
apt-get install gettext sqlite3 git build-essential python3 python3-dev python3-venv
martin.slouf's avatar
martin.slouf committed
82
83
84
85
# install those dependencies in order to compile required native
# Python packages for your platform
apt-get install libpq-dev

86
87
# clone the summer repo to the directory of your choice
git clone git@gitlab.com:martin.slouf/py-summer.git summer
martin.slouf's avatar
martin.slouf committed
88
89
cd summer

90
91
# create virtual environment with Python 3 interpreter and wheel
python3 -m venv .venv && . .venv/bin/activate && pip install wheel
martin.slouf's avatar
martin.slouf committed
92

93
# install dev requirements (see setup.py)
martin.slouf's avatar
martin.slouf committed
94
pip install -r requirements-dev.txt
martin.slouf's avatar
martin.slouf committed
95

martin.slouf's avatar
martin.slouf committed
96
# run tests
97
make clean test
martin.slouf's avatar
martin.slouf committed
98
99
100
#+END_SRC


martin.slouf's avatar
martin.slouf committed
101
102
103
* Files and Directories

#+BEGIN_EXAMPLE
martin.slouf's avatar
martin.slouf committed
104
# tree --charset ascii --dirsfirst -L 1 -F
martin.slouf's avatar
martin.slouf committed
105
.
martin.slouf's avatar
martin.slouf committed
106
107
108
109
110
111
112
|-- doc/               ... documentation (including release notes and examples)
|-- summer/            ... source code
|-- summer.egg-info/   ... PyPI info
|-- COPYING            ... GPL copy
|-- COPYING.LESSER     ... LGPL copy
|-- README.txt         ... this README document
`-- setup.py           ... Python distutils file
martin.slouf's avatar
martin.slouf committed
113
114
115
#+END_EXAMPLE


martin.slouf's avatar
martin.slouf committed
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
* Documentation

Reference documentation can be generated by /sphinx/ tool from source tree
and placed in file:doc/html/index.html.  It is also accessible through
project web page: http://py-summer.sourceforge.net/.

** Examples

Pack of commented examples is in [[file:summer/tests/examples]] directory.
Accessible once you download the source.


** Test suite

Once you checked out the sources, you can run tests from within the top
directory:

#+BEGIN_SRC sh
134
git clone git@gitlab.com:martin.slouf/py-summer.git summer
martin.slouf's avatar
martin.slouf committed
135
cd summer
136
make clean test
martin.slouf's avatar
martin.slouf committed
137
138
139
#+END_SRC

Commands above will clone official Git repository and execute popular
martin.slouf's avatar
martin.slouf committed
140
Python test runner ~nosetests~.
martin.slouf's avatar
martin.slouf committed
141
142


martin.slouf's avatar
martin.slouf committed
143
144
* Release notes

145
146
147
148
149
150
** 0.7.8.1

- fix summer import in case no LDAP, no ORM is required and extra libraries are
  missing


Martin Šlouf's avatar
Martin Šlouf committed
151
152
153
154
** 0.7.8

- remove un-necessary engine reference from `SessionFactory.Local`

155

156
157
158
159
160
161
162
** 0.7.7

- improve @session_scoped implementation so it not only closes (close()) the
  sqlalchemy session, but also deletes it (del) so any resources used are
  released and new access will produce new object


Martin Šlouf's avatar
Martin Šlouf committed
163
164
165
166
167
** 0.7.6.2

- improve AOP implementation to allow decorator chaining (if required)
- unfortunately, still using global namespace

168

Martin Šlouf's avatar
Martin Šlouf committed
169
170
171
172
** 0.7.6.1

- fix doc build

173

174
175
176
177
178
** 0.7.6

- upgrade libs to latest minor versions (ldap + sqlalchemy)
- review development dependencies (requirements-dev.txt)
- follow source code layout guidelines at https://www.pypa.io/, discussed https://blog.ionelmc.ro/2014/05/25/python-packaging/
Martin Šlouf's avatar
Martin Šlouf committed
179
- improve resource management
180

181

182
183
184
185
186
** 0.7.5.1

- slightly change behaviour of `Dao.get` method


Martin Šlouf's avatar
Martin Šlouf committed
187
188
189
190
** 0.7.5

- support for pytest framework
- migration to GitLab.com from SourceForge.net (still on SourceForge, but
Martin Šlouf's avatar
Martin Šlouf committed
191
  start using GitLab GIT as primary repo)
Martin Šlouf's avatar
Martin Šlouf committed
192
193
194
- ldap3 and sqlalchemy made optional deps (though using sqlalchemy unlocks
  most usefull features)

Martin Šlouf's avatar
Martin Šlouf committed
195
196
197

** 0.7.4.4.1

Martin Šlouf's avatar
Martin Šlouf committed
198
- hotfix for exception in __str__() method
Martin Šlouf's avatar
Martin Šlouf committed
199
200


Martin Šlouf's avatar
Martin Šlouf committed
201
202
203
204
205
** 0.7.4.4

- improve exception __str__ method


Martin Šlouf's avatar
Martin Šlouf committed
206
** 0.7.4.3
207
208
209
210

- Entity dao now supports domain classes derived from summer Entity and/or
  sqlalchemy declarative base

Martin Šlouf's avatar
Martin Šlouf committed
211

Martin Šlouf's avatar
Martin Šlouf committed
212
213
214
215
216
217
218
219
** 0.7.4.2

- add generic engine_kwargs to session provider that are passed to engine
  when created, one can manage pool size more easily for example
- removed default settings to engine 'pool_recycle=3600' -- use generic
  engine_kwargs to tune engine settings


Martin Šlouf's avatar
Martin Šlouf committed
220
** 0.7.4.1
221

Martin Šlouf's avatar
Martin Šlouf committed
222
223
- fix access to local bound connection -- use the connection of the
  underlying session, not a new connection from engine
224
225


226
227
228
229
230
231
232
233
234
235
236
** 0.7.4

- support for mixing declarative and classical SQLAlchemy mapping
- fix LDAP tests


** 0.7.3

- fix requirements specification


237
238
239
** 0.7.2

- producer/consumer refactor (producer/consumer/mediator)
Martin Šlouf's avatar
Martin Šlouf committed
240
241
242
- update copyright info
- fix rollback for nested @transactional decorators (original exception is
  not consumed anymore)
243
244


martin.slouf's avatar
martin.slouf committed
245
246
247
248
249
250
251
252
** 0.7.1

- upgrade dependencies (ldap3)
- specify minimum requirements in setup.py
- tested on Python 3.5+
- update Copyright information


martin.slouf's avatar
martin.slouf committed
253
254
** 0.7.0

martin.slouf's avatar
martin.slouf committed
255
- change summer configuration from text to Python config
martin.slouf's avatar
martin.slouf committed
256
257
- AOP implemntation redesign
- extensive use of @property annotation
martin.slouf's avatar
martin.slouf committed
258
- support for single SessionFactory and LdapSessionFactory (force singleton)
martin.slouf's avatar
martin.slouf committed
259
260


261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
** 0.6.2

- support for direct access of database connection
- virtual attributes can be accessed via getters (improves integration with
  IDE that can leverage Python 3 type declarations)
- some doc fixes


** 0.6.1

- fix package homepage url (https://py-summer.sourceforge.net/ -> http://py-summer.sourceforge.net/)


** 0.6.0

- Python 3 type annotations added
- setup.py changes according to https://packaging.python.org/en/latest/distributing.html
- Minor change in orm initialization (Context.orm_init() method)
- Generic Producer/Consumer framework
- New target (doc) to generate nice html documentation from source.
- Registered on PyPI


** 0.5.2

- migration to Python 3
- sources restructured to follow best practices of project layout
- source code formatting according to PEP 8
- broken compatibility in ldap
  removed base attribute in LDAP DAO session object (ie. self.session.base no longer works)
  added virtual attribute to LDAP DAO self.base instead of self.session.base


** 0.5.1

- Documentation update
- SourceForge.net release automation


** 0.5.0

- migration from subversion to git, thanks to great
  http://danielpocock.com/migrating-a-sourceforge-project-from-subversion-to-git
- several bug fixes
- improved logging


** 0.4.0

- Source documentation revisited.
- New unit tests added.
- Support for paging results using Filter domain object.
- (Ldap)Dao refactoring (new dao base class to support Dao without an entity).
- New utility class Printable for object pretty printing.


** 0.3.0

- some minor fixes (DaoSupport.query attribute, export of DaoException, ...)
- compatibility issues with SQLAlchemy 0.5, this is now the required version
- testing with real-world projects => first Beta release!

- known problems -- AOP -- calling proxied method (ie. decorated) from a
  non-proxy 'self' object (ie. calling private proxy method from non-proxied
  public method of the same object) does not invoke an advice

  workaround -- mark all your public methods with an annotation, do not rely on
  annotating private methods

  fix -- though possible, ugly; i will try to think up something better

- added assert module for various tests/checks -- ass.py
- utility method for exception stacktrace pretty printing -- even from aop
  code


** 0.2.2

- fix with DaoSupport.query attribute


** 0.2.1

- packaging bug fix regarding example 02_template_project
- dao support extended with more utility methods


** 0.2.0

- LDAP support added based on simillar concepts as
  SQL support (@ldapaop decorator, LDAP proxies)
- comprehensive example 02_template_project created


** 0.1.1

Just a minor release.  Added admin functionality to automate the
sourceforge release process + all the files where it makes sense now
include subversion 'Id' keyword.


** 0.1.0

This is initial release.  Contains no overall documentation, but one
commented example showing creation of application context, declarative
transactions and presents some basic hints and ideas.

It is a standard Python source distribution, so just unpack and install
with:

python setup.py install
martin.slouf's avatar
martin.slouf committed
372
373


martin.slouf's avatar
martin.slouf committed
374
* SourceForge notes                                                :noexport:
martin.slouf's avatar
martin.slouf committed
375

martin.slouf's avatar
martin.slouf committed
376
377
378
379
380
381
382
** Git

Project follows guidelines described in
http://nvie.com/posts/a-successful-git-branching-model/.

There 2 main branches:

martin.slouf's avatar
martin.slouf committed
383
384
1. /master/ holds current stable code
2. /develop/ for current development
martin.slouf's avatar
martin.slouf committed
385
386
387
388
389
390
391
392
393
394
395
396
397

All the development is done in /develop/ which is (once stable) merged to
/master/ -- usually just before release.

There is also a tag for each release.

Other (feature/hotfix) branches are being made as required.


** Project Web Site

- see [[file:Makefile]] target /sf-web-upload/
- web site is published automatically during release
martin.slouf's avatar
martin.slouf committed
398
- for manual publishing, execute:
martin.slouf's avatar
martin.slouf committed
399
400
401
402
403
404
405
406
407
408

#+BEGIN_SRC sh
SF_USERNAME=martinslouf make sf-web-upload
#+END_SRC

- there is also PyPI documentation page (http://pythonhosted.org/summer)
  that redirects to SourceForge, the page has been registered at PyPI
  manually (file:doc/pypi/index.html)


martin.slouf's avatar
martin.slouf committed
409
410
411
412
** Making a new release

*** Conventions

martin.slouf's avatar
martin.slouf committed
413
Early releases have been named by the date -- ~YYYY.MM.DD~ -- which was
martin.slouf's avatar
martin.slouf committed
414
415
cancelled -- releases are named by the version of the software.

martin.slouf's avatar
martin.slouf committed
416
Software version uses Python convention of ~MAJOR.MINOR.PATCH~.
martin.slouf's avatar
martin.slouf committed
417
418
419
420


*** During the release

martin.slouf's avatar
martin.slouf committed
421
- decide the release name (same as software version -- so nothing to decide :-)
Martin Šlouf's avatar
Martin Šlouf committed
422
423
- check/upgrade version as it is defined in
  - file:setup.py
424
  - [[file:src/summer/__init__.py]]
Martin Šlouf's avatar
Martin Šlouf committed
425
  - file:sphinx/source/conf.py (not anymore -- imported from summer)
martin.slouf's avatar
martin.slouf committed
426
- create release notes (new subsection in README.org)
martin.slouf's avatar
martin.slouf committed
427
- refresh documentation (this file =M-x org-ascii-export-to-ascii=)
martin.slouf's avatar
martin.slouf committed
428
429
430
431
432
- if everything seems fine:
  - merge with master
  - create a tag in SCM (git) for that release
- publish to ~sf.net~ and /PyPI/
- switch back to /develop/ branch
martin.slouf's avatar
martin.slouf committed
433
434
435
436
437


*** Automated process

There is an automated process for this -- the 'release' target of the
martin.slouf's avatar
martin.slouf committed
438
Makefile.  Executed from current branch, it:
martin.slouf's avatar
martin.slouf committed
439
440
441
442

1. runs clean & test
2. packages the project into Python source distribution

martin.slouf's avatar
martin.slouf committed
443
It also performs remote actions (confirmation is required):
martin.slouf's avatar
martin.slouf committed
444

martin.slouf's avatar
martin.slouf committed
445
446
3. creates a tag in SCM repository
4. uploads dist archive file via rsync to SourceForge FSR
martin.slouf's avatar
martin.slouf committed
447
5. uploads REAMDE.txt to SourceForge FSR
martin.slouf's avatar
martin.slouf committed
448
6. uploads web site via rsync to SourceForge WEB
martin.slouf's avatar
martin.slouf committed
449
7. registers new version at PyPI
martin.slouf's avatar
martin.slouf committed
450

martin.slouf's avatar
martin.slouf committed
451
452
453
Execute ~make release~ with:

#+BEGIN_SRC sh
Martin Šlouf's avatar
Martin Šlouf committed
454
VERSION=0.7.4.2 SF_USERNAME=martinslouf make release
martin.slouf's avatar
martin.slouf committed
455
#+END_SRC
martin.slouf's avatar
martin.slouf committed
456
457


martin.slouf's avatar
martin.slouf committed
458
459
460
461
462
463
464
465
** Project shell

#+BEGIN_SRC sh
# 1st create an instance
SF_USERNAME=martinslouf ssh $SF_USERNAME,py-summer@shell.sourceforge.net create
# 2nd login
SF_USERNAME=martinslouf ssh $SF_USERNAME,py-summer@shell.sourceforge.net
#+END_SRC
martin.slouf's avatar
martin.slouf committed
466
467


468
* TODO Todo list [2/11]
469
470
471
472

- [X] for nested transaction, there is double rollback -- and transaction
  is internally marked as not active, which causes the real exception to be
  hidden behind assert
martin.slouf's avatar
martin.slouf committed
473

474
475
476
477
478
- [X] review LDAP tests

- [ ] use twine to upload to PyPI

- [ ] migrate from sourceforge.net to gitlab.com
Martin Šlouf's avatar
Martin Šlouf committed
479
480
481
482
483
  - git migrated

- [ ] follow guidelines:
  - https://docs.pytest.org/en/latest/goodpractices.html#test-discovery
  - https://blog.ionelmc.ro/2014/05/25/python-packaging/
484

martin.slouf's avatar
martin.slouf committed
485
486
487
- [ ] intelligent context -- context that would inspect an object
  (@component annotation), auto-create proxy and auto-inject dependency
  (@inject annotation)
Martin Šlouf's avatar
Martin Šlouf committed
488
- [ ] lifecycle callbacks
martin.slouf's avatar
martin.slouf committed
489
490
491
492
493
494

#+BEGIN_SRC python
ContextListener:
    def context_initialized(context): pass
    def context_destroyed(context): pass

martin.slouf's avatar
martin.slouf committed
495
496
497
ComponentListener:
    def component_deployed(context): pass
    def component_undeployed(context): pass
martin.slouf's avatar
martin.slouf committed
498
#+END_SRC
499

500
501
502
- [ ] improve PCG -- it accepts an iterable and starts multiple threads to
  process it.  It can return an iterable (preserving the element order) of
  the results
martin.slouf's avatar
martin.slouf committed
503
504
- [ ] reimplement PC (and PCG) with multiprocessing support to side-step
  the GIL
martin.slouf's avatar
martin.slouf committed
505
- [ ] maybe use Python queue type instead of custom shared buffer?
Martin Šlouf's avatar
Martin Šlouf committed
506
- [ ] consider: discard sqlalchemy session on rollback?