Import from ASCII file

Import ASCII

AmiBroker has easy-to-use and flexible quotation import feature. This document describes advanced concepts of AmiBroker ASCII importer. Novice users should start with ASCII Import Wizard.

How does it work?

Quotation data may come from various sources so the format of the ASCII (i.e. text based) file may be much different from one source to another. To handle all those differences AmiBroker uses format definition commands that define the way the text information is interpreted by the ASCII importer. The format definition commands are keywords that begin with a dollar sign '$'. These commands may be embedded in the data file itself or, may be stored in the separate format definition file for multiple use. Storing format definition commands in separate file avoids the need to include the commands in every data file. The default format definition file name is "default.format". This file, all other ".format" files and "import.types" file (described later) should be stored in \Formats subdirectory of AmiBroker's current working directory. The defaults are overridden by any commands included (embedded) in the data file itself.

So, when you use the "Import from ASCII" menu, AmiBroker first looks for the format definition stored in "default.format" file and then parses the file you have chosen. If there is no "default.format" file then it uses internal defaults (described below).

You can modify "default.format" file to suit your needs. Moreover using OLE Automation (Win32 version) or ARexx (Amiga) interface you can specify the name of the format definition file which will be used instead of "default.format" file.

Format definition commands

The command keywords begin with a dollar sign '$''. Every line starting with command is interpreted in special way. Here is the list of commands recognized by AmiBroker's built-in importer. Bold letters mark keywords.

Command $ADDRESS Define company address

Arguments <string> address of company

Alias

Examples $ADDRESS "One Microsoft Way"

Command $AUTOADD Switch new ticker add mode

Arguments <number> 0 - do not add , 1 - add a new stock when non-existing ticker detected (default = 0)

Alias

Examples $AUTOADD 1

Command $ALLOWNEG Allow negative numbers in prices

Arguments <number>
0 - do not allow negative values (default), 1 - allow negative values in prices. This additionally switches off any checking for OHLC relationship so you can import any data into OHLC fields.

when $ALLOWNEG is NOT specified in the ASCII importer definition AmiBroker performs the following range checking and fixup on open, low and high prices
if( open == 0 ) open = close;
if( high < max( open, close ) ) high = max( open, close );
if( low == 0 ) low = min( open, close )

Alias

Examples $ALLOWNEG 1

Command $ALLOW99SECONDS Convert invalid second stamp

Arguments <onoff>
This flag works ONLY in conjunction with $TICKMODE 1 (see below for details)

$ALLOW99SECONDS set to 1 will convert all records with invalid seconds (i.e greater than 59)to 59s. So record stamped 16:29:70 will be treated as 16:29:59

Alias

Examples $ALLOW99SECONDS 1

Command $APPENDNAME append string to the ticker name (useful when you need to join several fields together to make unique stock symbol)

Arguments <string> string to append to the ticker symbol

Alias $APPENDTICKER

Examples

Command $BREAKONERR Define on-error behaviour

Arguments <number> 0 - to continue, 1 - to break import on error (default=0)

Alias

Examples $BREAKONERR 1

Command $CONT Define continuous quotations flag

Arguments <number> <0 or 1> - continuous quotations flag, this affects $AUTOADD 1 mode - if this is set, newly added stocks are switched to continuous quotation mode (this means enabling candlestick charts for example)

Alias

Examples $CONT 1

Command $CURRENCY Define symbol's currency

Arguments <string> Defines currency of symbol

Alias

Examples
$CURRENCY EUR

or

$FORMAT NAME, CURRENCY
$OVERWRITE 1
$AUTOADD 1

Command $DATE_DMY Define date

Arguments <number> The date in Canadian format (DD-MM-YY). If there is no argument given the date is taken from the file name (without an extension)

Alias $DATE_CDN

Examples $DATE_DMY 12-05-99
$DATE_CDN 12-05-1999

Command $DATE_MDY Define date

Arguments <number> The date in US format (MM-DD-YY). If there is no argument given the date is taken from the file name (without an extension)

Alias $DATE_USA

Examples $DATE_MDY 05/12/99
$DATE_USA 05/12/99

Command $DATE_YMD Define date

Arguments <number> The date in International format (YY-MM-DD). If there is no argument given the date is taken from the file name (without an extension)

Alias $DATE_INT

Examples $DATE_INT 99-05-12
$DATE_CDN 1999.05.12

Command $DEBUG Switch logging (debug) mode

Arguments <number> 0 - no error logging, 1 - log errors to "import.log" file (default=0)

Alias

Examples $DEBUG 1

Command $FORMAT Define line format (sequence and types of fields)

Arguments DATE_MDY date in US format: MM-DD-YY (alias: DATE_USA)

DATE_DMY date in Canadian format: DD-MM-YY (alias: DATE_CDN)

DATE_YMD date in International format: YY-MM-DD (alias: DATE_INT)

TIME time in HH:MM:SS or HH:MM or HHMM or HHMMSS format

NAME ticker name (alias: TICKER)

ALIAS symbol alias ($AUTOADD and $OVERWRITE modes only)

FULLNAME symbol full name ($AUTOADD and $OVERWRITE modes only)

OPEN open price

HIGH high price

LOW low price

CLOSE close price

ADJCLOSE split-adjusted close

This is provided to read adj. close column from Yahoo. Works only in conjunction with CLOSE field. When both CLOSE and ADJCLOSE are present in the ASCII format definition then importer calculates split factor by dividing ADJCLOSE/CLOSE. It then multiples OPEN, HIGH, LOW and CLOSE fields by this factor and divides VOLUME field by this factor. This effectively converts unadjusted prices to split adjusted prices. Split ratio gets locked once ADJCLOSE drops below 0.05.

OPENINT open interest

VOLUME volume

VOL1000 volume in thousands shares

VOLMIL volume in millions shares

VOLFACTOR volume factor (number of shares in a block) default =1

TURNOVER turnover

AUX1 AUX1 field (auxilliary data)

AUX2 AUX2 field (auxilliary data)

SKIP skip (ignore) field

MARKET specify a field that contains market ID (affects $AUTOADD and $OVERWRITE modes only)

GROUP specify a field that contains group ID (affects $AUTOADD and $OVERWRITE modes only)

WATCHLIST specify a field that contains watch list number (0-31) (affects $AUTOADD and $OVERWRITE modes only)

INFO specify a field with additional information (WSE specific: nk, ns, rk, rs, ok, os, zd, bd )

REDUCTION specify a field with reduction rate in percents (WSE specific)

ICB (new in 5.60)

specify ICB code

For example if your file looks as follows:
(format is symbol, full name, ICB code)
AAN,AARON'S INC,5375

Then to import it usign AmiBroker's import wizard use the following
$FORMAT Ticker,FullName,ICB
$OVERWRITE 1
$SEPARATOR ,
$CONT 1
$GROUP 255
$AUTOADD 1
$NOQUOTES 1

GICS
specify GICS code

For example if your file looks as follows:
(format is symbol, full name, gics sub industry code)
AAN,AARON'S INC,25504060

Then to import it usign AmiBroker's import wizard use the following
$FORMAT Ticker,FullName,GICS
$OVERWRITE 1
$SEPARATOR ,
$CONT 1
$GROUP 255
$AUTOADD 1
$NOQUOTES 1

INDUSTRY specify a field that contains industry ID (affects $AUTOADD and $OVERWRITE modes only)

INDUSTRYNAME
(new in 5.60) specifies a field that contains Industry Name. AmiBroker will check if given industry name already exists and if not, it will create a new Industry and assign imported stock to the industry specified. Also if SECTORNAME is specified, it will assign newly added industry to specified sector.

(affects $AUTOADD and $OVERWRITE modes only)

SECTORNAME
(new in 5.60) specifies a field that contains Sector Name. AmiBroker will check if given sector name already exists and if not, it will create a new Sector. Also if INDUSTRYNAME is specified, it will assign newly added industry to specified sector.

(affects $AUTOADD and $OVERWRITE modes only)

APPENDTICKER specify a field that contains string that should be appended to the ticker name (useful when you need to join several fields together to make unique symbol symbol)

MARGIN future contract margin deposit (positive value = dollars, negative value - percent of full value)

POINTVALUE future contract point value

ROUNDLOTSIZE round lot size (trading unit size)

TICKSIZE tick size

ADVISSUES number of advancing issues

ADVVOLUME volume of advancing issues

DECISSUES number of declining issues

DECVOLUME volume of declining issues

UNCISSUES number of unchanged issues

UNCVOLUME volume of unchanged issues

ADDRESS street address of company

CURRENCY specifies currency of symbol

WEBID specifies web ID

DIV_PAY_DATE
EX_DIV_DATE
LAST_SPLIT_DATE
LAST_SPLIT_RATIO
EPS
EPS_EST_CUR_YEAR
EPS_EST_NEXT_YEAR
EPS_EST_NEXT_QTR
FORWARD_EPS
PEG_RATIO
BOOK_VALUE (requires SHARES_OUT to be specified as well)
BOOK_VALUE_PER_SHARE
EBITDA
PRICE_TO_SALES (requires CLOSE to be specified as well)
PRICE_TO_EARNINGS (requires CLOSE to be specified as well)
PRICE_TO_BV (requires CLOSE to be specified as well)
FORWARD_PE (requires CLOSE to be specified as well)
REVENUE
SHARES_SHORT
DIVIDEND
ONE_YEAR_TARGET
MARKET_CAP (requires CLOSE to be specified as well - it is used to calculate shares outstanding)
SHARES_FLOAT
SHARES_OUT
PROFIT_MARGIN
OPERATING_MARGIN
RETURN_ON_ASSETS
RETURN_ON_EQUITY
QTRLY_REVENUE_GROWTH
GROSS_PROFIT
QTRLY_EARNINGS_GROWTH
INSIDER_HOLD_PERCENT
INSTIT_HOLD_PERCENT
SHARES_SHORT_PREV
FORWARD_DIV
OPERATING_CASH_FLOW
FREE_CASH_FLOW
BETA fundamental data fields. For more info read Using Fundamental Data

Alias

Examples $FORMAT TICKER DATE_MDY OPEN HIGH LOW CLOSE VOLUME
$FORMAT TICKER, DATE_INT, CLOSE, VOLUME
$FORMAT SKIP, TICKER, SKIP, SKIP, DATE_INT, OPEN, HIGH, LOW, CLOSE, TURNOVER

Command $FULLNAME Define full symbol name

Arguments <string> full symbol name

Alias

Examples $FULLNAME Apple Computer Inc.

Command $GICS Define GICS code (Global Industry Category System)

Arguments <number> this affects $AUTOADD 1 and $OVERWRITE 1 modes - if this is specified symbols are assigned to given GICS category

Alias

Examples Now you can import GICS symbol-code assignments using ASCII importer.
$FORMAT command now supports GICS code
and there is $GICS command for single-symbol files.
For example if your file looks as follows:
(format is symbol, full name, gics sub industry code)
AAN,AARON'S INC,25504060

Then to import it usign AmiBroker's import wizard use the following
$FORMAT Ticker,FullName,GICS
$OVERWRITE 1
$SEPARATOR ,
$CONT 1
$GROUP 255
$AUTOADD 1
$NOQUOTES 1

Command $GROUP Define group ID

Arguments <number> this affects $AUTOADD 1 mode - if this is specified, newly added symbols are assigned to group with given number.

Alias

Examples

Command $HYBRID Switch hybrid mode on/off

Arguments <number> 0 (off) or 1 (on). When this flag is set, you can combine quotations from multiple files - for example one file can contain only open prices and volume and the other file can contain high/low/close data. Useful especially for Warsaw Stock Exchange for combining the data from fixing and later continuous quotations.

Alias

Examples

Command $INDUSTRY Define industry ID

Arguments <number> this affects $AUTOADD 1 mode - if this is specified, newly added symbols are assigned to industry with given number.

Alias

Examples

Command $MARKET Define market ID

Arguments <number> this affects $AUTOADD 1 mode - if this is specified, newly added symbols are assigned to market with given number.

Alias

Examples

Command $NAME Define ticker name

Arguments <ticker> ticker name (symbol) (default = file name without path and extension)

Alias $TICKER

Examples $NAME AAPL
$TICKER MSFT

Command $NOQUOTES Switch quotation data mode

Arguments <number> 0 - (default) accept only quotation data (AmiBroker checks for non-zero prices and valid dates)
1 - switch off quotation data checking - this allows importing non-quotation data - for example only ticker and full names

Alias $TICKER

Examples $NAME AAPL
$TICKER MSFT

Command $OVERWRITE Switch overwrite mode on/off

Arguments <number> 0 - off, 1 - on. When overwrite mode is on then information provided by GROUP, MARKET, INDUSTRY, FULLNAME fields is overwritten for existing symbols (not only for newly added)

Alias

Examples $OVERWRITE 1

Command $PRICEFACTOR Define price factor

Arguments <number> the factor by which price data are multiplied (default = 1)

Alias

Examples $PRICEFACTOR 100

Command $RAWCLOSE2OI Put Raw Close price to OI field

Arguments <number> 0 - off, 1- on. (off by default) - causes that OpenInterest field gets assigned CLOSE (raw close) field value multiplied by 100

Alias

Examples $RAWCLOSE2OI 1

Command $RECALCSPLITS Recalculate splits

Arguments <number> 0 - off, 1- on. (off by default) causes that splits are recalculated by AmiBroker
by the algorithm that tries to construct correct adjusted price, based on inaccurate information provided by Yahoo.
Note that Yahoo provides only 2 decimal digits in adj. close field therefore the more adj. close approaches zero due to adjustements the error grows. The option $RECALCSPLITS 1 is intended to address this problem (at least partially).
It works as follows:
1. for each bar ratio ADJCLOSE/CLOSE is calculated
2. if the ratio changes in two consecutive bars by more than 10% it means that
split happened that bar. True split ratio is guessed by matching true fraction
in the format of X/Y, where X and Y = 1..9, to the change in ratios.
3. Then true split ratio is used to adjust all past bars until new split is detected.

Works only in conjunction with ADJCLOSE

Alias

Examples $RECALCSPLITS 1

Command $RECALCVOL Switch automatic index volume recalculation

Arguments <number> 0 - off, 1 - on (base index only), 2 - on (all indexes). When this is on AmiBroker calculates volumes for indexes based on assignments to markets and base indexes defined in Categories window

Alias

Examples $RECALCVOL 2

Command $RECALCAD Switch automatic advance/decline composite recalculation

Arguments <number>
0 - off, 1 - on. When this is on AmiBroker calculates numbers and volumes of issues advancing, declining and unchanged based on assignments to markets and base indexes defined in Categories window.

Alias

Examples $RECALCVOL 2

Command $ROUNDADJ Round split adjusted prices to given number of decimaldigits

Arguments <decimaldigits>
decimaldigits - causes split-adjusted prices (see above) to be rounded to 'decimaldigits' precision. By default no rounding is done

Works only in conjunction with ADJCLOSE

Alias

Examples $ROUNDADJ 2

Command $SEPARATOR Define field separator character

Arguments <separator char> the character used to separate data fields (default = space)

Alias

Examples $SEPARATOR ,
$SEPARATOR ;

Command $SKIPLINES Define how many lines to skip (ignore)

Arguments <number> number of lines to skip (default = 0)

Alias

Examples $SKIPLINES 1

Command $STRICT Switches on/off strict checking if Open, High, Low prices are greater than zero

Arguments <onoff> (default = 0)

Alias

Examples $STRICT 1

Command $TICKMODE Switches on/off tick mode

$TICKMODE is a special mode of importer that allows to import quotes that haveduplicate time stamps.
It makes two assumptions:
a) input data should come in the ascending time order (i.e. OLDER records first, LATEST records last)
b) input data should consist of entire tick history because importer will DELETE any existing quotes (to avoid creating multiple copies of the same ticks).

Once again: Turning on
$TICKMODE 1
will DELETE ANY QUOTES that already exist in the database and then will import all ticks from input data file.
You have been warned.

For example data files like this:

MOL,0,20050606,162959,16400.0000,16400.0000,16400.0000,16400.0000,2MOL,0,20050606,162959,16400.0000,16400.0000,16400.0000,16400.0000,11MOL,0,20050606,162959,16400.0000,16400.0000,16400.0000,16400.0000,40

Can be imported using the following definition file:

$FORMAT Ticker, Skip, Date_YMD, Time, Open, High, Low, Close, Volume
$SKIPLINES 1
$SEPARATOR ,
$CONT 1
$GROUP 255
$AUTOADD 1
$DEBUG 1
$TICKMODE 1

Sometimes it happens that input files have invalid timestamps (seconds > 59).

For example:

MOL,0,20050606,162970,16400.0000,16400.0000,16400.0000,16400.0000,2

Please take a closer look at first line shown in this example it has time:16:29:70 (you see 70 seconds !)

So I had to add a special flag to the importer that works around such data errors.

It is called $ALLOW99SECONDS 1 and will convert all records with invalid seconds (i.e greater than 59)to 59s.
So record stamped 16:29:70 will be treated as 16:29:59

Now for tick mode to work with such incorrect records you would need to add two lines to ASCII importer definition:

$TICKMODE 1
$ALLOW99SECONDS 1

Arguments <onoff> (default = 0)

Alias

Examples $TICKMODE 1

Command $TIMESHIFT Define intraday time shift used during import

Arguments <number> number of hours to shift date/time stamps (can be fractional)

Alias

Examples
$TIMESHIFT 2
; will shift 2 hours forward

$TIMESHIFT -11.5
; will shift 11 and half hour backward

Command $VOLFACTOR Define volume factor

Arguments <number> the factor by which volume data is multiplied (default = 1)

Alias

Examples $VOLFACTOR 10

Command $WATCHLIST Define watch list number

Arguments <number> this affects $AUTOADD 1 and $OVERWRITE 1 modes - if this is specified, newly added symbols are added to the watch list with given number.

Alias

Examples

Command $CLEANSECTORS Clean (wipe) existing sector/industry structure

Arguments <number> if this is turned on (1), existing sector/ industry structure will be deleted and initialized with Sector 0, 1, 2, 3...63/ Industry 0...255
This command should only be used in conjunction with SECTORNAME, INDUSTRYNAME $FORMAT fields to allow setting up fresh industry structure

Alias

Examples See example below (importing sector/industry structure)

Command $SORTSECTORS Sort sector/industry structure

Arguments <number> if this is turned on (1), sector/ industry structure will be sorted alphabetically after importing.
This command should only be used in conjunction with SECTORNAME, INDUSTRYNAME $FORMAT fields to allow setting up fresh industry structure

Alias

Examples See example below (importing sector/industry structure)

Command $USEONLYLOCALDB Switches "Use only local database" option for the symbol

Arguments <number> If data is fed by database plugin, using the ASCII importer to add any symbol causes these newly added symbol to have "Use only local database" flag turned on.
A new command:
$USEONLYLOCALDB 0
allows to turn this off (so newly added symbols have "use only local database" turned off)

This flag does NOT affect existing symbols.

Alias

Examples

Command $WEBID Define web ID

Arguments <string> web ID

Alias

Examples $WEBID aapl

Notes:

for DATE_xxx you can use ‘-‘, ‘/’ or ‘\’ as day/month/year separators. You can even omit separators at all if only you give a date in a 6 digit (YYMMDD, MMDDYY, DDMMYY) or 8 digit format (YYYYMMDD, MMDDYYYY, DDMMYYYY).
AmiBroker recognizes decimal as well as true fractions in price data. True fractions must follow the whole value after at least single space. For example you can specify: 5.33 or 5 1/3

AmiBroker is not limited to any kind of fraction, if you wish you can write even: 5 333/999

Comments

You can include comments in both format definition file and the data file(s). Each line starting with * (asterisk) or ; (semicolon) or # (hash) is treated as a comment and ignored by the ASCII importer.

Usage examples

What may look complicated from command list will become quite clear after some examples. So I will give you four examples of how to write format definition files. First example will show the definition for CSV (comma separated values) quotes available from Yahoo's finances site. Second example will show definition for Metastock ASCII file format. Third example shows definition for Omega SuperCharts ASCII file format. And fourth example will show the definition for s-files used by DM BOS (Polish brokerage company).

Yahoo CSV

The data from Yahoo's site looks as follows:

Date,Open,High,Low,Close,Volume 1-Feb- 0,104,105,100,100.25,2839600 31-Jan- 0,101,103.875,94.50,103.75,6265000 28-Jan- 0,108.1875,110.875,100.625,101.625,3779900

The first line gives us a hint about the meaning of the comma separated fields. First field will hold the date. The remaining fields will hold open, high, low, close prices and volume. Importer should skip the first line and parse all the remaining lines that hold just comma-separated data. Appropriate format definition file would look like this:

$FORMAT Date_DMY,Open,High,Low,Close,Volume $SKIPLINES 1 $SEPARATOR , $DEBUG 1 $AUTOADD 1 $BREAKONERR 1

$DEBUG switches on error logging to "import.log" file and $BREAKONERR will cause importer to stop after the first error found. $AUTOADD ensures that new ticker will be added to the database if it is missing. Well... you may ask: how does it know the ticker name? The answer is simple: if there is no field which defines the ticker name, the importer takes the file name (without path and extension) as a ticker. So if you are importing file "C:\My data\AAPL.CSV" AmiBroker will use "AAPL" as a ticker name.

Metastock ASCII

The data in Metastock ASCII format looks as follows:

<ticker>,<per>,<date>,<high>,<low>,<close>,<vol> AAP,D,1/17/2000,5483.33,5332.01,5362.3,0 AKS,D,1/17/2000,9868.45,9638.03,9687.62,0 FET,D,1/17/2000,3741.3,3540.2,3570.81,0

First field will hold the ticker name, second - time period ("D" means daily data), third - quotation date. The rest will hold high, low, close prices and volume. The importer should then skip the first line and parse all the remaining lines that hold just comma-separated data. Appropriate format definition file would look like this:

$FORMAT Ticker,Skip,Date_MDY,High,Low,Close,Volume $SKIPLINES 1 $SEPARATOR , $DEBUG 1 $AUTOADD 1 $BREAKONERR 1

Skip in $FORMAT defines a field which should be ignored by the importer.

Omega SuperCharts ASCII

The data in Omega SC ASCII format looks as follows:

ticker,date,open,high,low,close,vol AAP,20000117,5333.01,5483.33,5332.01,5362.3,3433450

This format is similar to previous ones, however the date is in YYYYMMDD format without separators between year, month and day part. AmiBroker, however, can handle such dates with ease. Appropriate format definition file would look like this:

$FORMAT Name,Date_Int,Open,High,Low,Close,Volume $SEPARATOR , $DEBUG 1 $SKIPLINES 1 $AUTOADD 1 $BREAKONERR 1

Skip in $FORMAT defines a field which should be ignored by the importer.

DMBOS S-files

The data in this format looks as follows:

0,29-02-00,12:05,MIDWIG,1069.1,,,+1.2,336002000, 0,29-02-00,12:05,NIF,48.6,,,+0.8,1763000, 0,29-02-00,12:05,WIG20,2300.3,,,+1.1,336002000, 0,29-02-00,12:05,WIG,21536.8,,,+0.2,336002000, 0,29-02-00,12:05,WIRR,2732.8,,,+1.6,16373000, 1,29-02-00,12:05,AGORA,144.00,,,+4.7,15802000, 1,29-02-00,12:05,AGROS,40.00,nk,72,+5.0,840000, 1,29-02-00,12:05,AMERBANK,28.00,,,+3.7,22000, 1,29-02-00,12:05,AMICA,41.50,nk,99,+2.2,564000,

This format is a little bit more complicated. For us useful fields are: 2nd - date, 4th - ticker, 5th - close price, 9th - the turnover value (close * volume). The remaining fields holds other information that is not useful for us. Appropriate format definition file would look like this:

$FORMAT Skip,Date_DMY,Skip,Name,Close,Skip,Skip,Skip,Turnover $SEPARATOR , $DEBUG 1

Importing Sector/Industry structure

Let's assume we have a text file with Stock tickers, Full names, Sector name and industry name listed line by line, as follows:

"DDD","3D Systems Corporation","Technology","Computer Software: Prepackaged Software"
"MMM","3M Company","Health Care","Medical/Dental Instruments"
"SVN","7 Days Group Holdings Limited","Consumer Services","Hotels/Resorts"
"AHC","A.H. Belo Corporation","Consumer Services","Newspapers/Magazines"
"AIR","AAR Corp.","Capital Goods","Aerospace"
"AAN","Aaron's, Inc.","Technology","Diversified Commercial Services"
"ABB","ABB Ltd","Consumer Durables","Electrical Products"

To import such file we use the following format definition:

$FORMAT Ticker, FullName,SectorName,IndustryName
$SEPARATOR ,
$AUTOADD 1
$NOQUOTES 1
$OVERWRITE 1
$CLEANSECTORS 1
$SORTSECTORS 1

$NOQUOTES 1 tells the importer that we will be importing non-quotation data. $AUTOADD 1/$OVERWRITE 1 is required to automatically add new symbols and overwrite existing symbol information. $CLEANSECTORS 1 wipes existing stock/industry structure prior to importing and $SORTSECTORS 1 - sorts sectors/industries after importing so they will be listed in alphabetical order in the Symbol window. $FORMAT command just specifies the order and types of field to import

AmiBroker will read such ASCII file one-by one, then it will check whenever given sector name/industry name already exists, if not - it will create new sector/industry. Then it will assign given symbol to specified sector/industry.

The result will be a database with new sector/industry structure being set up and symbols assigned to proper sectors and industries.

Described functionality is used to implement Tools->Update US symbol list and categories tool.

Default behaviour

When importing ASCII files, AmiBroker attempts to open "default.format" file (in the AmiBroker's directory) to obtain the format definition. If such file is missing the following default format is applied:

$FORMAT DATE_USA, OPEN, HIGH, LOW, CLOSE, VOLUME $SEPARATOR

This means that by default ASCII importer will use space character as a separator and will parse the following fields: date, open, high, low, close, volume. The file name (without path and extension) will be used as a ticker name. All other import parameters ($DEBUG,$AUTOADD, etc.) are set to zero.

User-definable file types and formats

Now AmiBroker can use not only default.format definition file but also other user-specified files. File types, filters and format definition files are specified in import.types file (example is included in the update package). Now user can prepare/modify import.types file with the description of supported ASCII formats and filters to use. The format of import.types file is:

<Descriptive name>|<File filter>|<definition file name>

Note vertical line characters between these three fields. Example import.types file looks as follows:

Default ASCII (*.*)|*.*|default.format Yahoo's CSV (*.csv)|*.csv|yahoo.format Metastock ASCII (*.mst)|*.mst|metastock.format Omega SC ASCII (*.txt)|*.txt|omega.format S-Files (s*.*)|s*.*|sfile.format C-Files (c*.*)|c*.*|cfile.format Sharenet DAT (*.dat)|*.dat|dat.format

If such file exists you will see your types in the "Files of type" combo-box and when you select one - appropriate filter will be used and after selecting some files and clicking OK - importer will use specified ".format" file.

In that way you can define as many text-based data formats as you like and AmiBroker will be able to "understand" them all.

Ticker aliases

Now each ticker can have an alias assigned, so the AmiBroker's built-in importers can recognize that security by both ticker symbol and alias names. This is useful when you are using two data sources that are using slightly different symbol naming convention or if you want to give the symbols more intuitive name while retaining the ability to use importers without problems.

GICS categorisation

GICS is global industry classification standard, see http://en.wikipedia.org/wiki/Global_Industry_Classification_Standard
for more details on GICS system.

GICS codes are from 2 to 8 digits. Such as 10 for energy sector or 351010 for "Health Care Equipment & supplies" industry.
The codes are fixed even if new classifications are added at some point in the future. It is important to understand that these codes work in hierarchical way.

NOTE: current databases DO NOT have GICS codes assigned to symbols.
As far as I know PremiumData http://www.premiumdata.net/ is planning to release AmiBroker-compatible database with GICS support.

AmiBroker now reads GICS.txt file from its installation folder. It contains GICS categories listed one by one in order of GICS code in the following format
GICS;Name;Description<CRLF>

GICS is numeric code from 2 digits upto 8 digits
Name is GICS category name
Description is GICS category description
These fields must be separated by semicolon
< CRLF> means carriage return/line feed characters (means "new line" - just press ENTER/RETURN key if you are editing with text editor)

There must be only one category per line in GICS.txt file

The default GICS.txt file is supplied already.

ICB categorisation

ICB stands for Industry Classification Benchmark (http://en.wikipedia.org/wiki/Industry_Classification_Benchmark).

AmiBroker allows also ICB 4-level classification system, but demo database does not have symbols classified according to that standard. You can find ICB classification codes in ICB.txt file inside AmiBroker folder.

NOTE: current databases DO NOT have ICB codes assigned to symbols.
ICB classification for NYSE stocks can be imported from http://www.nyse.com/indexes/nyaindex.csv

AmiBroker now reads ICB.txt file from its installation folder. It contains ICB categories listed one by one in order of ICB code in the following format
ICB;Name<CRLF>

ICB is numeric 4 digit code.
Name is ICB category name
These fields must be separated by semicolon
< CRLF> means carriage return/line feed characters (means "new line" - just press ENTER/RETURN key if you are editing with text editor)

There must be only one category per line in ICB.txt file

The default ICB.txt file is supplied already.

High resolution timestamps (milli- and micro-second)

Starting from version 6.14 ASCII importer supports microsecond resolution timestamps (HH:MM:SS.mmmuuu) where mmm - milliseconds 000..999, uuu - microseconds 000..999

Command	$ADDRESS	Define company address
Arguments	<string>	address of company
Alias
Examples	$ADDRESS "One Microsoft Way"

Command	$AUTOADD	Switch new ticker add mode
Arguments	<number>	0 - do not add , 1 - add a new stock when non-existing ticker detected (default = 0)
Alias
Examples	$AUTOADD 1

Command	$ALLOWNEG	Allow negative numbers in prices
Arguments	<number>	0 - do not allow negative values (default), 1 - allow negative values in prices. This additionally switches off any checking for OHLC relationship so you can import any data into OHLC fields. when $ALLOWNEG is NOT specified in the ASCII importer definition AmiBroker performs the following range checking and fixup on open, low and high prices if( open == 0 ) open = close; if( high < max( open, close ) ) high = max( open, close ); if( low == 0 ) low = min( open, close )
Alias
Examples	$ALLOWNEG 1

Command	$ALLOW99SECONDS	Convert invalid second stamp
Arguments	<onoff>	This flag works ONLY in conjunction with $TICKMODE 1 (see below for details) $ALLOW99SECONDS set to 1 will convert all records with invalid seconds (i.e greater than 59)to 59s. So record stamped 16:29:70 will be treated as 16:29:59
Alias
Examples	$ALLOW99SECONDS 1

Command	$APPENDNAME	append string to the ticker name (useful when you need to join several fields together to make unique stock symbol)
Arguments	<string>	string to append to the ticker symbol
Alias	$APPENDTICKER
Examples

Command	$BREAKONERR	Define on-error behaviour
Arguments	<number>	0 - to continue, 1 - to break import on error (default=0)
Alias
Examples	$BREAKONERR 1

Command	$CONT	Define continuous quotations flag
Arguments	<number>	<0 or 1> - continuous quotations flag, this affects $AUTOADD 1 mode - if this is set, newly added stocks are switched to continuous quotation mode (this means enabling candlestick charts for example)
Alias
Examples	$CONT 1

Command	$CURRENCY	Define symbol's currency
Arguments	<string>	Defines currency of symbol
Alias
Examples	$CURRENCY EUR or $FORMAT NAME, CURRENCY $OVERWRITE 1 $AUTOADD 1

Command	$DATE_DMY	Define date
Arguments	<number>	The date in Canadian format (DD-MM-YY). If there is no argument given the date is taken from the file name (without an extension)
Alias	$DATE_CDN
Examples	$DATE_DMY 12-05-99 $DATE_CDN 12-05-1999

Command	$DATE_MDY	Define date
Arguments	<number>	The date in US format (MM-DD-YY). If there is no argument given the date is taken from the file name (without an extension)
Alias	$DATE_USA
Examples	$DATE_MDY 05/12/99 $DATE_USA 05/12/99

Command	$DATE_YMD	Define date
Arguments	<number>	The date in International format (YY-MM-DD). If there is no argument given the date is taken from the file name (without an extension)
Alias	$DATE_INT
Examples	$DATE_INT 99-05-12 $DATE_CDN 1999.05.12

Command	$DEBUG	Switch logging (debug) mode
Arguments	<number>	0 - no error logging, 1 - log errors to "import.log" file (default=0)
Alias
Examples	$DEBUG 1

Command	$FORMAT	Define line format (sequence and types of fields)
Arguments	DATE_MDY	date in US format: MM-DD-YY (alias: DATE_USA)
	DATE_DMY	date in Canadian format: DD-MM-YY (alias: DATE_CDN)
	DATE_YMD	date in International format: YY-MM-DD (alias: DATE_INT)
	TIME	time in HH:MM:SS or HH:MM or HHMM or HHMMSS format
	NAME	ticker name (alias: TICKER)
	ALIAS	symbol alias ($AUTOADD and $OVERWRITE modes only)
	FULLNAME	symbol full name ($AUTOADD and $OVERWRITE modes only)
	OPEN	open price
	HIGH	high price
	LOW	low price
	CLOSE	close price
	ADJCLOSE	split-adjusted close This is provided to read adj. close column from Yahoo. Works only in conjunction with CLOSE field. When both CLOSE and ADJCLOSE are present in the ASCII format definition then importer calculates split factor by dividing ADJCLOSE/CLOSE. It then multiples OPEN, HIGH, LOW and CLOSE fields by this factor and divides VOLUME field by this factor. This effectively converts unadjusted prices to split adjusted prices. Split ratio gets locked once ADJCLOSE drops below 0.05.
	OPENINT	open interest
	VOLUME	volume
	VOL1000	volume in thousands shares
	VOLMIL	volume in millions shares
	VOLFACTOR	volume factor (number of shares in a block) default =1
	TURNOVER	turnover
	AUX1	AUX1 field (auxilliary data)
	AUX2	AUX2 field (auxilliary data)
	SKIP	skip (ignore) field
	MARKET	specify a field that contains market ID (affects $AUTOADD and $OVERWRITE modes only)
	GROUP	specify a field that contains group ID (affects $AUTOADD and $OVERWRITE modes only)
	WATCHLIST	specify a field that contains watch list number (0-31) (affects $AUTOADD and $OVERWRITE modes only)
	INFO	specify a field with additional information (WSE specific: nk, ns, rk, rs, ok, os, zd, bd )
	REDUCTION	specify a field with reduction rate in percents (WSE specific)
	ICB	(new in 5.60) specify ICB code For example if your file looks as follows: (format is symbol, full name, ICB code) AAN,AARON'S INC,5375 Then to import it usign AmiBroker's import wizard use the following $FORMAT Ticker,FullName,ICB $OVERWRITE 1 $SEPARATOR , $CONT 1 $GROUP 255 $AUTOADD 1 $NOQUOTES 1
	GICS	specify GICS code For example if your file looks as follows: (format is symbol, full name, gics sub industry code) AAN,AARON'S INC,25504060 Then to import it usign AmiBroker's import wizard use the following $FORMAT Ticker,FullName,GICS $OVERWRITE 1 $SEPARATOR , $CONT 1 $GROUP 255 $AUTOADD 1 $NOQUOTES 1
	INDUSTRY	specify a field that contains industry ID (affects $AUTOADD and $OVERWRITE modes only)
	INDUSTRYNAME	(new in 5.60) specifies a field that contains Industry Name. AmiBroker will check if given industry name already exists and if not, it will create a new Industry and assign imported stock to the industry specified. Also if SECTORNAME is specified, it will assign newly added industry to specified sector. (affects $AUTOADD and $OVERWRITE modes only)
	SECTORNAME	(new in 5.60) specifies a field that contains Sector Name. AmiBroker will check if given sector name already exists and if not, it will create a new Sector. Also if INDUSTRYNAME is specified, it will assign newly added industry to specified sector. (affects $AUTOADD and $OVERWRITE modes only)
	APPENDTICKER	specify a field that contains string that should be appended to the ticker name (useful when you need to join several fields together to make unique symbol symbol)
	MARGIN	future contract margin deposit (positive value = dollars, negative value - percent of full value)
	POINTVALUE	future contract point value
	ROUNDLOTSIZE	round lot size (trading unit size)
	TICKSIZE	tick size
	ADVISSUES	number of advancing issues
	ADVVOLUME	volume of advancing issues
	DECISSUES	number of declining issues
	DECVOLUME	volume of declining issues
	UNCISSUES	number of unchanged issues
	UNCVOLUME	volume of unchanged issues
	ADDRESS	street address of company
	CURRENCY	specifies currency of symbol
	WEBID	specifies web ID
	DIV_PAY_DATE EX_DIV_DATE LAST_SPLIT_DATE LAST_SPLIT_RATIO EPS EPS_EST_CUR_YEAR EPS_EST_NEXT_YEAR EPS_EST_NEXT_QTR FORWARD_EPS PEG_RATIO BOOK_VALUE (requires SHARES_OUT to be specified as well) BOOK_VALUE_PER_SHARE EBITDA PRICE_TO_SALES (requires CLOSE to be specified as well) PRICE_TO_EARNINGS (requires CLOSE to be specified as well) PRICE_TO_BV (requires CLOSE to be specified as well) FORWARD_PE (requires CLOSE to be specified as well) REVENUE SHARES_SHORT DIVIDEND ONE_YEAR_TARGET MARKET_CAP (requires CLOSE to be specified as well - it is used to calculate shares outstanding) SHARES_FLOAT SHARES_OUT PROFIT_MARGIN OPERATING_MARGIN RETURN_ON_ASSETS RETURN_ON_EQUITY QTRLY_REVENUE_GROWTH GROSS_PROFIT QTRLY_EARNINGS_GROWTH INSIDER_HOLD_PERCENT INSTIT_HOLD_PERCENT SHARES_SHORT_PREV FORWARD_DIV OPERATING_CASH_FLOW FREE_CASH_FLOW BETA	fundamental data fields. For more info read Using Fundamental Data
Alias
Examples	$FORMAT TICKER DATE_MDY OPEN HIGH LOW CLOSE VOLUME $FORMAT TICKER, DATE_INT, CLOSE, VOLUME $FORMAT SKIP, TICKER, SKIP, SKIP, DATE_INT, OPEN, HIGH, LOW, CLOSE, TURNOVER

Command	$FULLNAME	Define full symbol name
Arguments	<string>	full symbol name
Alias
Examples	$FULLNAME Apple Computer Inc.

Command	$GICS	Define GICS code (Global Industry Category System)
Arguments	<number>	this affects $AUTOADD 1 and $OVERWRITE 1 modes - if this is specified symbols are assigned to given GICS category
Alias
Examples	Now you can import GICS symbol-code assignments using ASCII importer. $FORMAT command now supports GICS code and there is $GICS command for single-symbol files. For example if your file looks as follows: (format is symbol, full name, gics sub industry code) AAN,AARON'S INC,25504060 Then to import it usign AmiBroker's import wizard use the following $FORMAT Ticker,FullName,GICS $OVERWRITE 1 $SEPARATOR , $CONT 1 $GROUP 255 $AUTOADD 1 $NOQUOTES 1

Command	$GROUP	Define group ID
Arguments	<number>	this affects $AUTOADD 1 mode - if this is specified, newly added symbols are assigned to group with given number.
Alias
Examples

Command	$HYBRID	Switch hybrid mode on/off
Arguments	<number>	0 (off) or 1 (on). When this flag is set, you can combine quotations from multiple files - for example one file can contain only open prices and volume and the other file can contain high/low/close data. Useful especially for Warsaw Stock Exchange for combining the data from fixing and later continuous quotations.
Alias
Examples

Command	$INDUSTRY	Define industry ID
Arguments	<number>	this affects $AUTOADD 1 mode - if this is specified, newly added symbols are assigned to industry with given number.
Alias
Examples

Command	$MARKET	Define market ID
Arguments	<number>	this affects $AUTOADD 1 mode - if this is specified, newly added symbols are assigned to market with given number.
Alias
Examples

Command	$NAME	Define ticker name
Arguments	<ticker>	ticker name (symbol) (default = file name without path and extension)
Alias	$TICKER
Examples	$NAME AAPL $TICKER MSFT

Command	$NOQUOTES	Switch quotation data mode
Arguments	<number>	0 - (default) accept only quotation data (AmiBroker checks for non-zero prices and valid dates) 1 - switch off quotation data checking - this allows importing non-quotation data - for example only ticker and full names
Alias	$TICKER
Examples	$NAME AAPL $TICKER MSFT

Command	$OVERWRITE	Switch overwrite mode on/off
Arguments	<number>	0 - off, 1 - on. When overwrite mode is on then information provided by GROUP, MARKET, INDUSTRY, FULLNAME fields is overwritten for existing symbols (not only for newly added)
Alias
Examples	$OVERWRITE 1

Command	$PRICEFACTOR	Define price factor
Arguments	<number>	the factor by which price data are multiplied (default = 1)
Alias
Examples	$PRICEFACTOR 100

Command	$RAWCLOSE2OI	Put Raw Close price to OI field
Arguments	<number>	0 - off, 1- on. (off by default) - causes that OpenInterest field gets assigned CLOSE (raw close) field value multiplied by 100
Alias
Examples	$RAWCLOSE2OI 1

Command	$RECALCSPLITS	Recalculate splits
Arguments	<number>	0 - off, 1- on. (off by default) causes that splits are recalculated by AmiBroker by the algorithm that tries to construct correct adjusted price, based on inaccurate information provided by Yahoo. Note that Yahoo provides only 2 decimal digits in adj. close field therefore the more adj. close approaches zero due to adjustements the error grows. The option $RECALCSPLITS 1 is intended to address this problem (at least partially). It works as follows: 1. for each bar ratio ADJCLOSE/CLOSE is calculated 2. if the ratio changes in two consecutive bars by more than 10% it means that split happened that bar. True split ratio is guessed by matching true fraction in the format of X/Y, where X and Y = 1..9, to the change in ratios. 3. Then true split ratio is used to adjust all past bars until new split is detected. Works only in conjunction with ADJCLOSE
Alias
Examples	$RECALCSPLITS 1

Command	$RECALCVOL	Switch automatic index volume recalculation
Arguments	<number>	0 - off, 1 - on (base index only), 2 - on (all indexes). When this is on AmiBroker calculates volumes for indexes based on assignments to markets and base indexes defined in Categories window
Alias
Examples	$RECALCVOL 2

Command	$RECALCAD	Switch automatic advance/decline composite recalculation
Arguments	<number>	0 - off, 1 - on. When this is on AmiBroker calculates numbers and volumes of issues advancing, declining and unchanged based on assignments to markets and base indexes defined in Categories window.
Alias
Examples	$RECALCVOL 2

Command	$ROUNDADJ	Round split adjusted prices to given number of decimaldigits
Arguments	<decimaldigits>	decimaldigits - causes split-adjusted prices (see above) to be rounded to 'decimaldigits' precision. By default no rounding is done Works only in conjunction with ADJCLOSE
Alias
Examples	$ROUNDADJ 2

Command	$SEPARATOR	Define field separator character
Arguments	<separator char>	the character used to separate data fields (default = space)
Alias
Examples	$SEPARATOR , $SEPARATOR ;

Command	$SKIPLINES	Define how many lines to skip (ignore)
Arguments	<number>	number of lines to skip (default = 0)
Alias
Examples	$SKIPLINES 1