Tags give the ability to mark specific points in history as being important
  • v0.5.1   OTraSys v0.5.1
    Release OTraSys v0.5.1

    OTraSys v0.5.1, released on October, 4th 2020

    The Open Trading System v0.5.1
    An open source framework to create trading systems

    changes from v0.5.0

    Intended as minor update, 0.5.1 brings a couple of big changes.
    First a major architecural change: separating the databases for quote data and for the system itself. This allows for one central quote repository, and multiple systems using that data (each a subset, or all the data with several parameter set- the possibilities are endless). This removes the need for redundant data if you run multiple systems on your machine.
    To ensure data integrity, OTraSys has read-only access to the quote repository. This means, v0.5.1 on the quote database has to be updated from outside the main system. As a starter, a new python script tools\update_quotes.py is provided under tools. Another new script, tools\setup_quote_db clears an existing quote database and sets it up, according to the configuration given.
    The second big change is a new symbolfile format. From now on the symbolfile is in csv format, all columns are separated by a semicolon ; and each entry can contain spaces (this has not been possible in former versions). Comments can still be used and are marked by a # (comments can span a whole line or the end of a line, after # a line won't be parsed further).
    The new symbolfile structe looks like this:

    #symbol;	identifier;	filename;   tradbl?;	ticks/pips;	contr.sz.;	weight;	cur;	type;	comment
    # ---------------------------------------------------------------------------------------------------------------------
    ADIDAS;		ADS.DE;		ADIDAS.csv;   1;	0.01;		1;		1;	EUR;	STOCK;	#
    AIRBUS;		AIR.DE;		AIRBUS.csv;   1;	0.01;		1;		1;	EUR;	STOCK;	#
    APPLE;		AAPL;		APPLE.csv;    1;	0.01;		1;		1;	USD;	STOCK;	#

    And there is the third somewhat bigger change, from now on you can not only use a symbol but also an identifier. This allows the central repository to hold different data sets for the same stock (useful for stocks with Type A/B, or stocks in several currencies, or futures with different due dates, or whatever you want).

    Minor improvements include more interaction with the database from the command line, with the new cli option C/--set-cash the amount of the current account´s cash can be adjusted.

    Description

    New features

    • new command line options:
      • -C, --set-cash AMOUNT set cash for current account to AMOUNT
    • changed/new config file options:
      • the config file now supports [sections], indicated by brackets
      • due to the separated databases, the database settings have doubled: DB_QUOTES_NAME, DB_QUOTES_HOSTNAME, DB_QUOTES_USERNAME, ... and DB_SYSTEM_NAME, DB_SYSTEM_HOSTNAME, DB_SYSTEM_USERNAME
    • database changes:
      Please note that the changes mentioned below break compatibility with prior program versions!
      • changed string length for symbol in all tables to 40 (was: 10)
      • separated databse into a system database and a quote database

    Bugfixes

    • SIGNAL_REGIME_FILTER in general did not work as the flag was not evaluated due to a regression
    • SIGNAL_REGIME_FILTER=TNI in special now working again, the calculation was broken since code refactoring during v0.4.0 development (separate problem from above bugfix)
    • minor cleanups fixing memory leaks
  • v0.5.0   v0.5.0
    Release v0.5.0

    OTraSys v0.5.0, released on March, 20th 2020

    The Open Trading System v0.5
    An open source framework to create trading systems

    changes from v0.4.0


    Description

    We hereby proudly present version 0.5 of OTraSys, the Open Trading System Framework.
    This release marks a huge milestone for the software, for several reasons. The most obvious one being the new name. Since the software is now mature enough for productive testing, we changed the name to avoid possible problems due to the name conflict with another IP. Additionally, the old name was more of a joke that somehow stuck through development.

    This release is a continuation from the complete under-the-hood overhaul that v0.4 brought. But aside some bug fixes and improved stability, also a couple of important features made it into the release:

    • The incorporation of trading costs: For backtests the system now considers the trading costs for each transaction. There are 2 new configfile options (TRANSACTION_COST_FIX and TRANSACTION_COST_PERCENTAGE) by which you can model the cost structure of your broker. As trading costs overtime can render a system unprofitable over time, this is an important aspect for reliable backtests.
    • The performance of the system can now be evaluated in more detail, as the system keeps track of your cash, equity, risk-free-equity (the value of all current positions valued at their current stop loss) for each trading day. Additionally to the still-present statistics summary (invoke the software with --report or -r at the command line) this allows more detailed insights in the performance. For convienence, a shell script plot_performance.sh under /tools will extract the data from the database and plot a performance graph.
    • The way how stoplosses are tracked was completely redone. If you configure the system (SL_SAVE_DB = true) to save SL to database, it will generate a stop loss record for each active position for each trading day and write that to the database at the end. The default is SL_SAVE_DB = false, as this generates a huge amount of data which is often not neccessary (with the exception if you are currently developing a new system and want to test the effect of different stop loss strategies).

    Please note that the system is currently working only for stocks correctly.
    For FOREX, Futures and CFD the P/L calculation is broken, but will be redone in future releases.
    The database record for all stoplosses was redone and is working again.

    New features

    • added a shell script which will generate a performance chart as seen in documentation sample performance
    • for STOCKS: implemented transaction costs (see new config file options), users can now define a fixed amount or a percentage of the value as fee (or mix both)
    • changed/new config file options:
      • new:
        • TRANSACTION_COSTS_FIX: fixed amount fee in account currency
        • TRANSACTION_COSTS_PERCENTAGE: fee as percentage of position´s value (see configuration)
      • SL_SAVE_DB: save all SL records within database? (true/false)
      • added SL_TYPE = atr which implements a single ATR based stop, simpler than Chandelier (which still works)
      • changed 2 parameters SL_CHANDELIER_ATR_PERIOD and SL_CHANDELIEUR_ATR_FACTOR to SL_ATR_PERIOD and SL_ATR_FACTOR as they apply to both Chandelier and ATR stops
    • database changes:
      Please note that the changes mentioned below break compatibility with prior program versions!
      • added field fee to orderbook_daily and sum_fees', all_time_hightoaccount`

      • added new table performance_record that tracks account information for each day

      • renamed table stoploss_daily to stoploss_record, now using the fields as defined in the documentation (see table stoploss_record)

    Bugfixes

    • Reworked drawdown calculation, it is now based on performance records. A drawdown now is a state when the current total (cash + equity) is lower than the all time high total.
    • Fixed an (apparently older) bug that a position would not be considered for a check of SL, if a position (right before that one in the portfolio) was sold on the same day
    • cosmetics: for long-only (and short-only) systems, the statistics report (--report/-r) will show the long (or short) only numbers and not an empty table with nans for the other side
    • fixed a wrong "Covariance matrix inversion failed!" warning which was in fact also printed if it didn't fail
    • not a bug, but a long standing nuisance: within doxygen documentation, all methods are now linked to their implementation, so developers can see the interface of methods
  • v0.4.0   v0.4.0
    80282366 · prepare for release ·
    Release v0.4.0

    ichinscratchy v0.4.0 "itsbeenawhile", released on February, 16th 2020

    changes from v0.3.0

    Description

    This looks like a maintenance release, as the list of new features is quite short. However I rewrote ~85% of the internal code to make it cleaner, leaner and better extensible/modifyable. By doing so the execution speed was also greatly enhanced. Internal representation of quotes, indicators, markets, ... now follows an object oriented model. Alltogether this release marks the transition from a pure imple- mentation of an Ichimoku trend following system to a framework flexible enough to create any kind of trading system.

    New features

    • for STOCKS: position sizing is now performed based on stop loss, current balance, risk free equity and RISK_PER_POSITION; CFD and FUTURES still pending
    • database changes:
      Please note that the changes mentioned below break compatibility with prior program versions!
      • added field description to ichimoku_daily_signals
      • table quotes_daily: renamed changes to returns
      • portfolio now has the following structure: symbol char(10) NOT NULL, markettype char(6) NOT NULL, currency_market char(3) NOT NULL, buyquote_market decimal(16,6) NOT NULL, buydate_market date NOT NULL, buydaynr_market int NOT NULL, buyquote_account decimal(16,6) NOT NULL, buydate_account date NOT NULL, buydaynr_account int NOT NULL, buyquote_signal decimal(16,6) NOT NULL, buydate_signal date NOT NULL, buydaynr_signal int NOT NULL, currquote_market decimal(16,6) NOT NULL, currdate_market date NOT NULL, currdaynr_market int NOT NULL, currquote_account decimal(16,6) NOT NULL, currdate_account date NOT NULL, currdaynr_account int NOT NULL, type char(5) NOT NULL, signalname char(30) NOT NULL, stoploss decimal(16,6) NOT NULL, initial_sl int NOT NULL, pos_size decimal(10,2), quantity decimal(10,2), trading_days int, current_value decimal(16,6), invested_value decimal(16,6), risk_free_value decimal(16,6), p_l decimal(16,6), risk_free_p_l decimal(16,6),p_l_percent decimal(10,2)
    • changed/new config file options:
      • new: ACCOUNT_CURRENCY_DIGITS. This parameter sets the nr of significant digits for account currency.
      • new: the parser of the symbolfile now accepts whitespace and tabs between fields
      • new: the symbolfile has a new column called weights, where you can enter manual weights for this market. used in combination with PORTFOLIO_ALLOCATION = fixed
      • new: PORTFOLIO_RETURNS_PERIOD. This parameter defines the period for mean returns
      • new: PORTFOLIO_RETURNS_UPDATE. update mean returns every X periods
      • new: RISK_FREE_RATE which specifies rate of returns for 0 risk investment
      • new: POS_SIZE_CAP which specifies a cap size for new positions: x% of current balance + risk free Equity
      • removed: PORTFOLIO_SPEARMAN_PERIOD, PORTFOLIO_COR_INTV_X_MIN/MAX, PORTFOLIO_COR_INTV_X_WEIGHT, PORTFOLIO_PN_INTV_X_MIN/MAX
      • modified: PORTFOLIO_ALLOCATION: removed spearman, added kelly and fixed as valid options:
        • kelly calculate weights by optimizing expected value of returns
        • fixed use fixed weights as provided in the symbolfile
    • changed/ new program start parameters:
      • added command line option --orderbook/-o which skips signal execution and prints out orderbook and account info from database
      • added command line option --account/-a which displays current account information to terminal
      • command line option --portfolio/-p now displays account info also

    Bugfixes

    • completely removed the code that did portfolio allocation based on Spearman correlation. As I know now, the reasoning behind that implementation was flawed, and honestly it did not work very well (if you keep the occasional curve fitting aside)
    • fixed a bug that prevented ICHI_CHIKOU_CONFIRM filter to work properly in many situations
    • fixed a ton of memory leaks

    other changes

    • skipped signals are now printed to terminal only if -v/ --verbose command line option is set
    • changed makefile to use -O3 and -funroll-loops optimization
  • v0.3.0   ichinscratchy v0.3.0
    Release v0.3.0

    ichinscratchy v0.3, released on December, 17th 2017

    changes from v0.2.1

    New features:

    • changed symbol file contents and layout, new symbol file includes information that is needed for money management:
      # Indices =========================================================================================
      #symbol     filename        tradeable?      ticks/pips     contract size    cur     type        # comment
      #--------------------------------------------------------------------------------------------------
      DAX         DAX.csv             1           0.01              1             EUR     CFDFUT      # DAX30
      DJI         DOW.csv             1           0.01              1             USD     CFDFUT      # Dow Jones
      # Futures =========================================================================================
      #symbol     filename        tradeable?      ticks/pips     contract size    cur     type        # comment
      #--------------------------------------------------------------------------------------------------
      #BuFu        BuFu.csv           1           0.01              1             EUR     CFDFUT      # Euro Bund Future
      # Forex ===========================================================================================
      #symbol     filename        tradeable?      ticks/pips     contract size    cur     type        # comment
      #--------------------------------------------------------------------------------------------------
      EURJPY      EURJPY.csv          1           0.001           100000          EUR     CFDCUR      # Euro/ Japanese Yen
      EURUSD      EURUSD.csv          0           0.00001         100000          EUR     CFDCUR      # Euro/US Dollar
          ```
      comments still start with a '#', can be as a separate comment line or at the end of a line.  
      Basic parsing and sanity checks are implemented, still user has to put some thought into the config. This change
      breaks compatibility of the symbol file with former releases. 
    • changed/ new program start parameters:
      • added command line option -c/--configfile which specifies the name of the configuration under /config. This allows to run several systems/configurations from within one main folder (adjust database name accordingly)
      • when using verbose option -v/--verbose the program now spits out the account info (balance, equity and risk- free equity) each day, together with the already known buy-/sell signals.
      • when using the portfolio option -p/--portfolio the program shows the current account info (balance, equity and risk- free equity) in account currency, for each position current value (and risk free value) is shown
      • when using the report option -r/--report all wins/losses are now calculated in account currency
    • database changes: Please note that the changes mentioned below break compatibility with prior program versions!
      • changed all float fields to decimal(x,y). With float under certain circumstances strange behaviour occured, as floats are not represented/stored exactly. Fixed decimals are exact representations instead.
      • added new table account that stores some values which are needed to calculate profit/losses, your current cash, equity and such. The table hosts variables with their corresponding values. As in theory this table can store arbitrary variables and values, the following variables are currently needed by the program (and expected in the table):
        • balance (current free cash in account)
        • equity (free cash plus current value of positions- if sold right NOW)
        • risk_free_equity (free cash plus current value of positions if all their stop losses were hit NOW)
        • virgin_flag (initialized with 1 when creating db, valid until first position is bought- after that 0)
      • added new table currency_translation that holds all information for translations from/to account currency
      • added new fields to indicators_daily: ADX, regime_filter
      • the tables quotes_daily, indicators_daily, ichimoku_daily and ichimoku_daily_signals now have an additional column: daynr, which holds the date as "days since 1900-01-01"
      • the table orderbook_daily now has 2 price fields: price which holds the price in the symbol´s currency and cost_per_item which holds the buying price in account´s currency
      • the table portfolio now has 3 price fields: price_buy and price_last which hold the buying and the latest price in market´s currency and cost_per_item which holds the buying costs in accounts currency
    • changed/new config file options:
      • new: ACCOUNT_CURRENCY. This parameter sets the currency of your account. Currently implemented and valid values are:
        • EUR (Euro)
        • USD (US Dollar)
        • JPY (Japanese Yen)
      • new: INDI_ADX_PERIOD. This option specifies the period, over which the ADX shall be smoothed. Basically, the ADX is an exponential average of the DMI, which in turn contains the ATR, an exponential average of the True Range. INDI_ADX_PERIOD specifies the number of periods in those exponential moving averages. Default 14
      • new: SIGNAL_REGIME_FILTER. This option switches the market regime filter on (and choses which one to use). In sideways/non- trending markets signals will not be executed, resulting in lower trading frequency but higher quality signals. Following options are valid:
        • none (Turn market regime filter off)
        • ADX (Use Average Directional Index)
        • TNI (Trend Normality Indicator by @tasciccac)
      • new: SIGNAL_REGIME_ADX_THRESH. When SIGNAL_REGIME_FILTER is set to ADX, this option specifies the threshhold under which a market will be considered as sideways (=trendless). In this case signals in this specific market won't be executed. Default is 30.
      • new: STARTING_BALANCE, which sets starting account balance at begin of backtest/live trading
      • new: RISK_PER_POSITION which sets risk of a new position in % of current equity. In conjunction with the calculated initial stop loss this determines the size of the new position (in terms of how many units will be bought).
    • under /tools a new script sql2csv was created. This little helper reads in the config, executes a sql script within the same directory and gives the results in a .csv file. Two example files get_adx.sql (which gets date, close and ADX for a specific symbol) and export_orderbook.sql (which replaces the former export_trades.sh script) are already included (and likely to get more in the future)
    • startmysql.sh now expects a command line option -c followed by config file name (located under ../config)
    • small update to statistics report: now also display overall win/loss

    Bugfixes

    • fixed a (longtime) bug that caused execution to exclude the last day of available data
  • v0.2.1
    Release v0.2.1

    ichinscratchy v0.2.1, released on April, 9th 2017

    changes from v0.2.0

    New features:

    • database changes: added new fields to indicators_daily: HH_atr_period, LL_atr_period
    • added config file option SIGNAL_EXECUTION_SUNDAYS. If set to false, signals that occur on sundays are not executed. This can easily happen depending on users local time/date: When some markets open on monday morning, in other regions of the world it is still sunday evening
    • colored terminal output: errors -> red, warnings -> yellow, cyan -> SL, green -> signals
    • streamlined/shortened terminal output for signal/SL execution
    • colored html output: green -> uptrend/long, red -> downtrend/short, blue -> not trending
    • integrated persistence1D library to find local minima and maxima in price data (see https://people.mpi-inf.mpg.de/~weinkauf/notes/persistence1d.html)
    • added config file option ICHI_KUMO_CONFIRM. If set to true for a Kumo breakout to be valid the price has to make a higher high/lower low on close above/below previous (horizontal) reaction high/ reaction low. This drastically improves signal quality of Kumo breakouts at the cost of making a Kumo breakout a pretty rare event. This filter is the first benefit of integrating the persistence1d library into ichinscratchy codebase (more to come :))

    Bugfixes:

    • Chandelier Stop now correctly use the SL_CHANDELIER_ATR_PERIOD nr. of days to determine highest highs/lowest lows (see database change). Prior to this fix the hh26/ll26 was used.
    • fixed situation when initial SL (Chandelier) is above price for long or below price for short position. This can happen in situation when there were sharp volatility changes in past. Fallback is to use the percentage stop for those situations
  • v0.2.0
    bdeaebb4 · Release of v0.2.0 YAY \m/ ·
    Release v0.2.0

    ichinscratchy v0.2.0, released on March, 5th 2017

    ================================================= changes from v0.1.0

    New features:

    • implemented portfolio management using spearman correlation
    • new config option PORTFOLIO_ALLOCATION which selects the portfolio allocation model, currently equal (all position sizes are equal) and spearman (position size of new trade is determined by spearman correlation with existing portfolio entries)
    • added config option PORTFOLIO_SPEARMAN_PERIOD which determines the period used for calculating Spearmans Rho
    • added a bunch of config options PORTFOLIO_CORR_... and PORTFOLIO_PN_... to tailor position sizing based on correlation
    • database changes: portfolio now has the column pos_size; orderbook_daily column size was renamed to quantity and additional column pos_size was added ==> THIS BREAKS DATABASE COMPATIBILTY WITH versions < v0.2.0!!
    • added command line option -p/--portfolio to skip signal evaluation/execution and print out current portfolio to database
    • added average (mean) and median holding days to orderbook analysis/ reporting function (use --report or -r command line option)
    • export_trades.sh and startmysql.sh in /tools now parse the config file under /config, no more hardcoded db credentials in scripts
    • release version now comes with example DAX30 data file for users to start experimenting with
    • migrated source code documentation to doxygen-compatible style --> full reference in html/pdf now available under /docs

    Bugfixes:

    • if run twice without updating the quotes (with a couple of days to execute), the program would sell positions based on the last SL in database. If days to execute date back long enough, it selled even positions which were bought later
    • when running backwards, the trend filter ICHI_CHIKOU_CONFIRM did not work properly
    • if portfolio contained more than 1 position with same SL, for each day the SL was hit only 1 SL was executed
    • when reporting statistics (-r/--report or config option) and there is no trade in db --> print message instead of crashing :)
  • v0.1.0   initial release v0.1.0
    Release v0.1.0

    Well... it compiles! It runs! It does create ichimoku signals! (They are correct!!! :)) It follows the configures stop loss strategy! It can do backtests! Errm. Ok, it is my very first release.