On a separate note, we’ve run into a problem with FsHtmlDoc – it blows up when trying to document the Xaye.Core classes. The bundled API docs for the time being only include the Xaye.Math API.  We are looking for an alternative to FsHtmlDoc.

Next we’ll add support for importing historical stock data from Yahoo Finance and EODData‘s historical data service (EODData provides historical intraday trade bars). Then we can finally start doing some interesting things with the data.

Constants are for storing constant floating point values. Earlier they were generic but constraining them to floats simplified the code (floats cover most cases). Formulas are for storing strings that resolve to variables at run time. For example “diff(cpi)/lag(cpi)”, would evaluate to the percentage change in CPI. Whenever the CPI variable changes so would the evaluated value of the formula. Note that formulas cannot be resolved from the Xaye.Core API, but only from the Xaye runtime (coming soon).

Variables are ordered series of data. They support float, decimal, long, datetime, string, tradebar, and opaque data types. A trade bar is a simple record of decimals: close, open, low, high, and volume (unit32) where only the close value is required. We didn’t include a field for the trade interval (the period covered by the bar) since it could be stored in the item’s documentation (see below). An opaque type is a wrapper around a byte array. Users can pass a raw array or an instance of any serializable type to the constructor.

VariableGroups are a collection of variables with their data rows aligned. Groups can be sorted and users can access individual rows. We’ll be adding the ability to add, delete and filter rows (observations), and iterate over them. We’ve also added support to read/write text delimited files into/from variable groups. We’ll be supporting Excel (xls and xlsx) soon for reading and writing variable groups.

Each of the four types expose a Details object that contains the name of the item, a description field, a documentation field, and a list of string key-value attributes. We chose string valued attributes for simplicity and they are most the common use case. Users that want to store binary data for attributes (such as an image or PDF file) will have to base64 their data.

Delimited files can only support variables, so for local storage we’ll be using SQLite. The current SQLite storage code isn’t optimized but uses the generic ADO.NET interfaces. The reason is that we can quickly test other SQL databases as data stores using the same code (MS SQL Server and SQL Azure will be next). We’ll eventually write optimized code for each supported database. Support for STATA files, SAS XPORT files, and NoSQL databases are on the long-term to-do list.

On another note, we are dropping support for Linux/Mono for the time being. Mono chokes on the Variable class and we’ll adding back support once we figure out a workaround.

Times are in milliseconds and the benchmark was ran on a six core, 3.3 GHz i7 running x64 Windows 7. Relative performance was computed as PP/MKL.

The simplest way to use the Xaye GA is use the solve method. It only requires a cost/fitness function and bounds on the variables of the cost function. For an example, lets minimize:       for -10 < x < 10

let cost = fun (x:float[]) ->(x.[0]*x.[0]+x.[0]) * Math.Cos(x.[0])
let bound = {lower = -10.0; upper = 10.0}
let bounds = [| bound; |]
let status = solve cost bounds
printfn "%A" status
{Iteration = 69;
 Best = ([|9.620350903|], -100.2237551);
 NoImprovement = 25;
 Termination = NoImprovement;}

The solve function returns a Status record that contains the number of iterations, best solution found, how many iterations since we found the best solution (NoImprovement), and why the optimization terminated. In this case, solve needed 69 iterations to find the minimum and it terminated early because it was no longer improving on the best solution (we default to a maximum of a 1000 iterations and stop if we there was no  improvement after 25 iterations). It uses the our defaultMate function (top x chromosome selection, pairing parents using a rank squared weighted roulette, and a heuristic blend crossover), the uniformMutate function, and generates a random starting population of 20. These defaults work well for relatively simple functions.

Lets try a little more difficult cost function:      for -10 < x < 10.  The minimum is at x,y=0.

let cost = fun (x:float[]) ->
   let x0 = Math.Abs(x.[0])
   let x1 = Math.Abs(x.[1])
   x0 - 10.0*Math.Cos(Math.Sqrt(10.0*x0)) + x1 - 10.0*Math.Cos(Math.Sqrt(10.0*x1))
let bound = {lower = -10.0; upper = 10.0}
let bounds = [| bound; |]
let status = solve cost bounds
printfn "%A" status
{Iteration = 41;
 Best = ([|3.510642794; 0.007216450758|], -15.48906593);
 NoImprovement = 25;
 Termination = NoImprovement;}

We got stuck at a local minimum. Instead of using the solve function, we can use the defaultEvolve function. This uses the same default mating and mutation methods as solve, but allows us tweak the optimization options. Lets increase the size of the population, decrease the selection rate (the percentage of the current generation to use for mating and to copy into the next generation), and increase the number of no improvement iterations. We will now sample more of the solution space but at a cost of a longer running time. As you can see below, we now find the global minimum.

let options = {DefaultOptions with PopulationSize=1000; SelectionRate=0.1; NoImprovement=50 }
let status = defaultEvolve options cost bounds
printfn "%A" status
{Iteration = 92;
 Best = ([|1.585990353e-17; -4.292601412e-18|], -20.0);
 NoImprovement = 50;
 Termination = NoImprovement;}

We also have a fully customizable evolve function that allows users to build any type of GA.

type Fitness = float
type Gene = float
type Chromosome = Gene[]
type Population = Chromosome[]
type EvaluatedPopulation = (Chromosome*Fitness)[]
type Parents = Chromosome*Chromosome
type Cost = Chromosome -> Fitness
type Mate = EvaluatedPopulation -> Population
type Mutate =  Population -> Population
type Convergence = Options -> Status -> EvaluatedPopulation -> bool
evolve : Options -> Cost -> Mate -> Mutate -> Convergence -> Population -> Status

For example, to create a GA where we replace the full population each generation, use Haupt’s extrapolation crossover, a 3 chromosome tournament for selecting parents, and use a normally distributed mutation method (gene + random normal(0,1) ), we’d:

let cost = fun (x:float[]) ->
   let x0 = Math.Abs(x.[0])
   let x1 = Math.Abs(x.[1])
   x0 - 10.0*Math.Cos(Math.Sqrt(10.0*x0)) + x1 - 10.0*Math.Cos(Math.Sqrt(10.0*x1))
let bound = {lower = -10.0; upper = 10.0}
let bounds = [| bound; bound; |]
let replaceAll (options:Options) (bounds:Bounds[]) (population:EvaluatedPopulation) : Population =
   let pairs = options.PopulationSize/2
   let pairing = pairTournament3 options.Random pairs
   let crossover = extrapolationCrossover options.Random
   population |> pairing |> crossover

use norm = new Normal01()
let mutation = mutateNormal options norm bounds
let initial = generateInitial options bounds
let status = evolve DefaultOptions cost mate mutation defaultConvergence initial

defaultConvergence just checks to see if we’ve reached the maximum number of iterations or that the best solution hasn’t improved in options.NoImprovement number of iterations.

Next we’ll implements parts of chapter 5: hybrid GA (mix the GA with a local optimizer), multiple objective functions, and a parallel GA . And tweak the current implementation to make it more “functional” – still getting the hang of functional programming.

Another tweak is the Samples method. It now returns an infinite sequence of random numbers (seq<float>).  Just don’t do

use rng = new UniformRng(0.0, 1.0)
rng.Samples() |> Seq.average

I’m not sure how useful it will be and we might remove it in the future (it was trivial to add).

This is NOT an open source project since it uses commercial and closed source components. Because of this, you may NOT redistribute Xaye (see EULA.txt for more details). Xaye will be free to use, but you cannot distribute it. We are making the code that we write available under the 2-clause BSD license.

Current status:
We started working on the Xaye core library (IO, linear algebra, statistics, etc.). Not much there yet – so far some basic linear algebra, estimation, and statistics routines. We decided not to use Math.NET Numerics for the core library for two reasons.  First, we wanted to use the F# PowerPack  matrix classes. Second, I’m no longer interested in supporting Silverlight or purely managed code scenarios. This allows us to make use of Intel’s Math Kernel Library (MKL) without having to write the corresponding  functionality in managed code.

Over the next two weeks we’ll be adding a genetic algorithm component - working through “Practical Genetic Algorithms“. Then we’ll move on to the scripting language.

You can follow Xaye’s development here. Links to the current builds are on sidebar and most of the source code is at GitHub.