Final Report

This is the final report on my work with the interval package during Google Summer of Code 2017. This whole blog have been dedicated to the project and by reading all posts you can follow my work from beginning to end.

The work has been challenging and extremely fun! I have learned a lot about interval arithmetic, the Octave and Octave-forge project, and also how to contribute to open-source in general. I have found the whole Octave community to be very helpful and especially I want to thank my mentor, Oliver Heimlich, and co-mentor, Kai Torben Ohlhus, for helping me during the project.

Here I will give a small introduction to Octave and the interval package for new readers and a summary of how the work has gone and how you can run the code I have actually contributed with.

Octave and the Interval Package

Octave, or GNU Octave, is a free program for scientific computing. It is very similar to Matlab and its syntax is largely compatible with it. Octave comes with a large set of core functionality but can also be extended with packages from Octave forge. These add new functionality, for example image processing, fuzzy logic or more statistic functions. One of those packages is the interval package which allows you to compute with interval arithmetic in Octave.

Summary of the Work

The goal with the project was to improve the Octave-forge interval package by implementing support for creating, and working with, N-dimensional arrays of intervals.

The work has gone very well and we have just released version 3.0.0 of the interval package incorporating all the contributions I have made during the project.

The package now has full support for working with N-dimensional arrays in the same way you do with ordinary floating point numbers in Octave. In addition I have also fixed some bugs not directly related to N-dimensional arrays, see for example bug #51783 and #51283.

During the project I have made a total of 108 commits. I have made changes to 332 of the packages 666 files. Some of these changes have only been changes in coding style or (mainly) automated adding of tests. Not counting these I have, manually, made changes to 110 files.

If you want to take a look at all the commits I have contributed with the best way is to download the repository after which you can see all the commits from GSoC with

hg log -u joeldahne@hotmail.com -d "2017-06-01 to 2017-08-29"

Unfortunately I have not found a good way of isolating commits from a specific period and author on sourceforge where the package is hosted. Instead you can find a list of all commits at the end of this blog post.

The NEWS-file from the release of version 3.0.0 is also a pretty good overview of what I have done. While not all of the changes are a result of GSoC quite a lot of them are.

Running the Code

As mentioned above we have just released version 3.0.0 of the interval package. With the new release it is very easy to test the newly added functionality. If you already have Octave installed the easiest way to install the package is with the command “pkg install -forge interval”. This will install the latest release of the package, at the time of writing this is 3.0.0 but that will of course change in the future. You can also download version 3.0.0 directly from Octave-forge.

If you want you can also download the source code from the official repository and test it with "make run" or install it with "make install". To download the repository, update to version 3.0.0 an run Octave with the package on Linux use the following

 hg clone http://hg.code.sf.net/p/octave/interval octave-interval
 cd octave-interval
 hg update release-3.0.0
 make run

A Package for Taylor Arithmetic

The task took less time than planned so I had time to work upon a project depending on my previous work. I started to work on a project for Taylor arithmetic, you can read my blog post about it. I created a proof of concept implementation as part of my application for Google Summer of Code and I have now started to turn that into a real package. The repository can be found here.

It is still far from complete but my goal is to eventually add it as a package at Octave-Forge. How that goes depends mainly on how much time I have to spend on it the following semesters.

If you want to run the code as it is now you can pull the repository and then run it with "make run", this requires that Octave and version 3.0.0 (or higher) of the interval package is installed.

List of Commits

Here is a list of all 108 commits I have done to the interval package

https://sourceforge.net/p/octave/interval/ci/68eb1b3281c9
summary:     Added Swedish translation for package metadata

https://sourceforge.net/p/octave/interval/ci/52d6a2565ed2
summary:     @infsupdec/factorial.m: Fix decoration (bug #51783)

https://sourceforge.net/p/octave/interval/ci/cf97d56a8e2a
summary:     mpfr_function_d.cc: Cast int to octave_idx_type

https://sourceforge.net/p/octave/interval/ci/5ed7880c917f
summary:     maint: Fix input to source

https://sourceforge.net/p/octave/interval/ci/637c532ea650
summary:     @infsupdec/dot.m: Fix decoration on empty input

https://sourceforge.net/p/octave/interval/ci/51425fd67692
summary:     @infsup/postpad.m, @infsup/prepad.m, @infsupdec/postpad.m, @infsupdec/prepad.m: Corrected identification of dimension for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/b4e7f1546e36
summary:     mpfr_function_d.cc, mpfr_vector_dot_d.cc: Fixed bug when broadcasting with one size equal to zero

https://sourceforge.net/p/octave/interval/ci/02db5199932c
summary:     doc: NEWS.texinfo: Info about vectorization for nthroot and pownrev

https://sourceforge.net/p/octave/interval/ci/197d033fc878
summary:     @infsup/pownrev.m, @infsupdec/pownrev.m: Support for vectorization of p

https://sourceforge.net/p/octave/interval/ci/7f8d1945264c
summary:     @infsup/nthroot.m, @infsupdec/nthroot.m, mpfr_function_d.cc: Support for vectorization of n

https://sourceforge.net/p/octave/interval/ci/e841c37383d6
summary:     doc: NEWS.texinfo: Summarized recent changes from GSoC

https://sourceforge.net/p/octave/interval/ci/bc3e9523d42a
summary:     mpfr_vector_dot_d.cc: Fixed bug when broadcasting with one size equal to zero

https://sourceforge.net/p/octave/interval/ci/0160ff2c2134
summary:     doc: examples.texinfo: Updated example for the latest Symbolic package version

https://sourceforge.net/p/octave/interval/ci/f37e1074c3ce
summary:     @infsup/*.m, @infsupdec/*.m: Added missing N-dimensional versions of tests

https://sourceforge.net/p/octave/interval/ci/eb6b9c01bf6a
summary:     @infsup/*.m, @infsupdec/*.m: N-dimensional versions of all ternary tests

https://sourceforge.net/p/octave/interval/ci/de71e0ad0a6e
summary:     @infsup/*.m, @infsupdec/*.m: N-dimensional versions of all binary tests

https://sourceforge.net/p/octave/interval/ci/9440d748b4aa
summary:     @infsup/*.m, @infsupdec*.m: N-dimensional version of all unary tests

https://sourceforge.net/p/octave/interval/ci/cb5b7b824400
summary:     @infsup/powrev2.m: Fixed bug when called with vector arguments

https://sourceforge.net/p/octave/interval/ci/a3e8fdb85b97
summary:     @infsup/pownrev.m, @infsupdec/pownrev.m: Reworked vectorization test

https://sourceforge.net/p/octave/interval/ci/aae143789b81
summary:     @infsup/pown.m: Added support for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/f05814665570
summary:     @infsup/nthroot.n: Reworked vectorization test

https://sourceforge.net/p/octave/interval/ci/c4c27574e331
summary:     @infsup/nthroot.m, @infsupdec/nthroot.m: Clarified that N must be scalar

https://sourceforge.net/p/octave/interval/ci/3bfd4e7e6600
summary:     @infup/pow.m, @infsupdec/pow.m: Fixed bug when called with vector arguments

https://sourceforge.net/p/octave/interval/ci/e013c55a4e2a
summary:     @infsup/overlap.m, @infsupdec/overlap.m: Fixed formatting of vector test.

https://sourceforge.net/p/octave/interval/ci/d700a1ad4855
summary:     doc: Modified a test so that it now passes

https://sourceforge.net/p/octave/interval/ci/2404686b07fb
summary:     doc: Fixed formatting of example

https://sourceforge.net/p/octave/interval/ci/c078807c6b32
summary:     doc: SKIP an example that always fail in the doc-test

https://sourceforge.net/p/octave/interval/ci/598dea2b8eb8
summary:     doc: Fixed missed ending of example in Getting Started

https://sourceforge.net/p/octave/interval/ci/66ca24edfba4
summary:     Updated coding style for all infsupdec-class functions

https://sourceforge.net/p/octave/interval/ci/4c629b9000fc
summary:     Updated coding style for all infsup-class functions

https://sourceforge.net/p/octave/interval/ci/3898a7ad5d93
summary:     Updated coding style for all non-class functions

https://sourceforge.net/p/octave/interval/ci/f911db83df14
summary:     @infsupdec/dot.m: Fixed wrong size of decoration when called with two empty matrices

https://sourceforge.net/p/octave/interval/ci/8e6a9e37ee40
summary:     Small updates to documentation and comments for a lot of function to account for the support of N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/a2ef249d54ae
summary:     doc: A small update to Examples, the interval Newton method can only find zeros inside the initial interval

https://sourceforge.net/p/octave/interval/ci/76431745772f
summary:     doc: Updates to Getting Started, mainly how to create N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/9821eadca12b
summary:     doc: Small updates to Preface regarding N-dimensional arrays and fixed one link

https://sourceforge.net/p/octave/interval/ci/0f1aadf5f13a
summary:     ctc_intersect.m, ctc_union.m: Fixed bugs when used for vectorization and when called with 0 or 1 output arguments

https://sourceforge.net/p/octave/interval/ci/0e01dc19dc75
summary:     @infsup/sumsq.m: Updated to use the new functionality of dot.m

https://sourceforge.net/p/octave/interval/ci/2fba2056ed31
summary:     @infsup/dot.m, @infsupdec/dot.m, mpfr_vector_dot_d.cc: Added support for N-dimensional vectors. Moved all vectorization to the oct-file. Small changes to functionality to mimic how the sum function works.

https://sourceforge.net/p/octave/interval/ci/05fc90112ea9
summary:     ctc_intersect.m, ctc_union.m: Added support for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/2a4d1e9fa43e
summary:     @infsup/fsolve.m: Added support for N-dimensional arrays. Fixed problem with the function in the example. Improved performance when creating the cell used in vectorization.

https://sourceforge.net/p/octave/interval/ci/7a96c346225a
summary:     @infsup/disp.m: Fixed wrong enumeration of submatrices

https://sourceforge.net/p/octave/interval/ci/933117890e45
summary:     Fixed typo in NEWS.texinfo

https://sourceforge.net/p/octave/interval/ci/67734688adb1
summary:     @infsup/diag.m: Added description of the previous bug fix in the NEWS file

https://sourceforge.net/p/octave/interval/ci/07ebc81867cd
summary:     @infsup/diag.m: Fixed error when called with more than 1 argument

https://sourceforge.net/p/octave/interval/ci/554c34fb3246
summary:     @infsup/meshgrid.m, @infsupdec/meshgrid.m: Removed these functions, now falls back on standard implementation, also updated index

https://sourceforge.net/p/octave/interval/ci/4bad2e7451f5
summary:     @infsup/plot.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/5e0100cdc25f
summary:     @infsup/plot3.m: Small change to allow for N-dimensional arrays as input

https://sourceforge.net/p/octave/interval/ci/695a223ccbb7
summary:     @infsupdec/prod.m: Added support for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/55f570c13f01
summary:     @infsup/prod.m: Added support for N-dimensional arrays. Removed short circuit in simple cases.

https://sourceforge.net/p/octave/interval/ci/445d7e5150aa
summary:     @infsup/sum.m, @infsupdec/sum.m, mpfr_vector_sum_d.cc: Added support for N-dimensional vectors. Moved all vectorization to the oct-file. Small changes to functionality to mimic Octaves standard sum function.

https://sourceforge.net/p/octave/interval/ci/f27ad212735e
summary:     @infsup/fminsearch.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/19724b3f581e
summary:     mpfr_function_d.cc: Finalized support for N-dimensional arrays with binary functions and added support for it with ternary functions.

https://sourceforge.net/p/octave/interval/ci/023e2788e445
summary:     midrad.m: Added tests for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/617484113019
summary:     @infsupdec/infsupdec.m: Added full support for creating N-dimensional arrays and added tests

https://sourceforge.net/p/octave/interval/ci/f1c28a3d48b7
summary:     @infsup/subset.m, @infsupdec/subset.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/8e9990e3ed13
summary:     @infsup/strictsubset.m, @infsupdec/strictsubset.m: Fixed coding style and updated documentation

https://sourceforge.net/p/octave/interval/ci/5ca9b26b580d
summary:     @infsup/strictprecedes.m, @infsupdec/strictprecedes.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/1c76721003f1
summary:     @infsup/sdist.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/e044dea7c820
summary:     @infsup/precedes.m, @infsupdec/precedes.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/d48ca9206299
summary:     @infsup/overlap.m, @infsupdec/overlap.m: Fixed coding style and updated documentation

https://sourceforge.net/p/octave/interval/ci/1ad6b47f9335
summary:     @infsup/issingleton.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/11dccbcfd97e
summary:     @infsup/ismember.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/ade5da12cf88
summary:     @infsup/isentire.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/9fc874b71533
summary:     @infsup/isempty.m, @infsupdec/isempty.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/0c5c4eaf263b
summary:     @infsup/iscommoninterval.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/c980da83b634
summary:     @infsup/interior.m, @infsupdec/interior.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/f67d78651aee
summary:     @infsup/idist.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/70a389bd59f5
summary:     @infsup/hdist.m: Fixed coding style and updated documentation.

https://sourceforge.net/p/octave/interval/ci/e34401d2a6d6
summary:     @infsup/sin.m, @infsupdec/sin.m: Added workaround for bug #51283

https://sourceforge.net/p/octave/interval/ci/fea4c6516101
summary:     @infsup/gt.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/ce973432d240
summary:     @infsup/ge.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/a435986d4b0c
summary:     @infsup/lt.m, @infsupdec/lt.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/23fa89e3c461
summary:     @infsup/le.m, @infsupdec/le.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/730397b9e339
summary:     @infsup/disjoint.m, @infsupdec/disjoint.m: Updated documentation

https://sourceforge.net/p/octave/interval/ci/682ad849fc98
summary:     mpfr_function_d.cc: Added support for N-dimensional arrays for unary functions. Also temporary support for binary functions.

https://sourceforge.net/p/octave/interval/ci/9661e0256c51
summary:     crlibm_function.cc: Added support for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/0ec3d29c0779
summary:     @infsup/infsup.m: Fixed documentation and added missing line continuation

https://sourceforge.net/p/octave/interval/ci/ebb00e763a46
summary:     @infsup/disp.m: Fixed documentation

https://sourceforge.net/p/octave/interval/ci/1ab2749da374
summary:     @infsup/size.m: Fixed documentation

https://sourceforge.net/p/octave/interval/ci/507ca8478b72
summary:     @infsup/size.m: Fixes to the documentation

https://sourceforge.net/p/octave/interval/ci/667da39afece
summary:     nai.m: Small fix to one of the tests

https://sourceforge.net/p/octave/interval/ci/3eb13f91065a
summary:     hull.m: Fixes according to Olivers review

https://sourceforge.net/p/octave/interval/ci/20d784a6605d
summary:     @infsup/display.m: Vectorized loop

https://sourceforge.net/p/octave/interval/ci/fab7aa26410f
summary:     @infsup/disp.m: Fixes according to Olivers review, mainly details in the output

https://sourceforge.net/p/octave/interval/ci/4959566545db
summary:     @infsup/infsup.m: Updated documentation and added test for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/b25fbf0ad324
summary:     @infsup/infsup.m: Fixed coding style

https://sourceforge.net/p/octave/interval/ci/486a73046d5e
summary:     @infsup/disp.m: Updated documentation and added more tests for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/b6a0435da31f
summary:     exacttointerval.m: Uppdated documentation and added tests for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/bdad0e6da132
summary:     @infsup/intervaltotext.m, @infsupdec/intervaltotext.m: Updated documentation and added tests for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/55a3e708aef4
summary:     @infsup/intervaltotext.m: Fixed coding style

https://sourceforge.net/p/octave/interval/ci/536b0c5023ee
summary:     @infsup/subsref.m, @infsupdec/subsref.m: Added tests for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/c9654a4dcd8d
summary:     @infsup/size.m: Added support for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/636709d06194
summary:     @infsup/end.m: Added support for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/e6451e037120
summary:     nai.m: Added support for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/837cccf1d627
summary:     @infsup/resize.m, @infsupdec/resize.m: Added support for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/11cb9006b9ea
summary:     @infsup/reshape.m, @infsupdec/reshape.m: Added support for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/c276af6c42ae
summary:     @infsup/prepad.m, @infsupdec/prepad.m: Added small parts to the documentation and tests for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/c92c829dc946
summary:     @infsup/postpad.m, @infsupdec/postpad.m: Added small parts to the documentation and tests for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/0f8f6864123e
summary:     @infsup/meshgrid.m, @infsupdec/meshgrid.m: Added support for outputting 3-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/b7faafef6030
summary:     @infsup/cat.m, @infsupdec/cat.m: Added support for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/d416a17a6d3d
summary:     hull.m: Added support for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/031831f6bdfd
summary:     empty.m, entire.m: Added support for N-dimensional arrays

https://sourceforge.net/p/octave/interval/ci/2816b68b83c4
summary:     @infsup/display.m: Added support for displaying high dimensional arrays

https://sourceforge.net/p/octave/interval/ci/79c8dfa8ac54
summary:     @infsup/disp.m: Added support for displaying high dimensional arrays

https://sourceforge.net/p/octave/interval/ci/cc87924e52ac
summary:     @infsup/disp.m: Fixed coding style

https://sourceforge.net/p/octave/interval/ci/29a6b4ecda2a
summary:     @infsupdec/infsupdec.m: Temporary fix for creating high dimensional arrays

https://sourceforge.net/p/octave/interval/ci/039abcf7623d
summary:     @infsupdec/infsupdec.m: Fixed coding style

The Final Polish

We are preparing for releasing version 3.0.0 of the interval package and this last week have mainly been about fixing minor bugs related to the release. I mention two of the more interesting bugs here.

Compact Format

We (Oliver) recently added support for "format compact" when printing intervals. It turns out that the way to determine if compact format is enabled differs very much between different version of Octave. There are at least three different ways to get the information.

In the older releases (< 4.2.0 I believe) you use "get (0, "FormatSpacing")" but there appear to be a bug for version < 4.0.0 for which this always return "loose".

For the current tip of the development branch you can use "[~, spacing] = format ()" to get the spacing.

Finally in between these two version you use "__compactformat__ ()".

In the end Oliver, probably, found a way to handle this mess and compact format should now be fully supported for intervals. The function to do this is available here https://sourceforge.net/p/octave/interval/ci/default/tree/inst/@infsup/private/__loosespacing__.m.

Dot-product of Empty Matrices

When updating "dot" to support N-dimensional arrays I also modified it so that it behaves similar to Octaves standard implementation. The difference is in how it handles empty input. Previously we had

> x = infsupdec (ones (0, 2));
> dot (x, x)
ans = 0×2 interval matrix

but with the new version we get

> dot (x, x)
ans = 1×2 interval vector
   [0]_com   [0]_com

which is consistent with the standard implementation.

In the function we use "min" to compute the decoration for the result. Normally "min (x)" and "dot (x, x)" returns results of the same size (the dimension along which it is computed is set to 1), but they handle empty input differently. We have

> x = ones (0, 2);
> dot (x, x)
ans =
   0   0
> min (x)
ans = [](0x2)

This meant that the decoration would be incorrect since the implementation assumed they always had the same size. Fortunately the solution was very simple. If the dimension along which we are computing the dot-product is zero. the decoration should always be "com". So just adding a check for that was enough.

You could argue that "min (ones (0, 2))" should return "[inf, inf]" similarly to how many of the other reductions, like "sum" or "prod", return their unit for empty input. But this would most likely be very confusing for a lot of people. And it is not compatible with how Matlab does it either.

Updates on the Taylor Package

I have also had some time to work on the Taylor package this week. The basic utility functions are now completed and I have started to work on functions for actually computing with Taylor expansions. At the moment there are only a limited amount of functions implemented. For example we can calculate the Taylor expansion of order 4 for the functions $\frac{e^x + \log(x)}{1 + x}$ at $x = 5$.

## Create a variable of degree 4 and with value 5
> x = taylor (infsupdec (5), 4)
x = [5]_com + [1]_com X + [0]_com X^2 + [0]_com X^3 + [0]_com X^4
## Calculate the function
> (exp (x) + log (x))./(1 + x)
ans = [25.003, 25.004]_com + [20.601, 20.602]_com X + [8.9308, 8.9309]_com X^2 + [2.6345, 2.6346]_com X^3 + [0.59148, 0.59149]_com X^4

Improving the Automatic Tests

Oliver and I have been working on improving the test framework used for the interval package. The package shares a large number of tests with other interval packages through an interval test framework that Oliver created. Here is the repository.

Creating the Tests

Previously these tests were separated from the rest of the package and you usually ran them with help of the Makefile. Now Oliver has moved them to the m-files and you can run them, together with the other tests for the function, with test @infsup/function in Octave. This makes it much easier to test the functions directly.

In addition to making the tests easier to use we also wanted to extend them to not only test scalar evaluation but also vector evaluation. The test data, input ad expected output, is stored in a cell array and when performing the scalar testing we simply loop over that cell and run the function for each element. The actual code looks like this (in this case for plus)

%!test
%! # Scalar evaluation
%! testcases = testdata.NoSignal.infsup.add;
%! for testcase = [testcases]'
%!   assert (isequaln (...
%!     plus (testcase.in{1}, testcase.in{2}), ...
%!     testcase.out));
%! endfor

For testing the vector evaluation we simply concatenate the cell array into a vector and give that to the function. Here is what that code looks like

%! # Vector evaluation
%! testcases = testdata.NoSignal.infsup.add;
%! in1 = vertcat (vertcat (testcases.in){:, 1});
%! in2 = vertcat (vertcat (testcases.in){:, 2});
%! out = vertcat (testcases.out);
%! assert (isequaln (plus (in1, in2), out));

Lastly we also wanted to test evaluation of N-dimensional arrays. This is done by concatenating the data into a vector and then reshape that vector into an N-dimensional array. But what size should we use for the array? Well, we want to have at least three dimensions because otherwise we are not really testing N-dimensional arrays. My solution was to completely factor the length of the vector and use that as size, testsize = factor (length (in1)), and if the length of the vector has two or fewer factors we add a few elements to the end until we get at least three factors. This is the code for that

%!test
%! # N-dimensional array evaluation
%! testcases = testdata.NoSignal.infsup.add;
%! in1 = vertcat (vertcat (testcases.in){:, 1});
%! in2 = vertcat (vertcat (testcases.in){:, 2});
%! out = vertcat (testcases.out);
%! # Reshape data
%! i = -1;
%! do
%!   i = i + 1;
%!   testsize = factor (numel (in1) + i);
%! until (numel (testsize) > 2)
%! in1 = reshape ([in1; in1(1:i)], testsize);
%! in2 = reshape ([in2; in2(1:i)], testsize);
%! out = reshape ([out; out(1:i)], testsize);
%! assert (isequaln (plus (in1, in2), out));

This works very well, except when the number of test cases is to small. If the number of test is less than four this will fail. But there are only a handful of functions with that few tests so I fixed those independently.

Running the tests

Okay, so we have created a bunch of new tests for the package. Do we actually find any new bugs with them? Yes!

The function pow.m failed on the vector test. The problem? In one place $\&\&$ was used instead of $\&$. For scalar input I believe these behave the same but they differ for vector input.

Both the function nthroot.m and the function pownrev.m failed the vector test. Neither allowed vectorization of the integer parameter. For nthroot.m this is the same for standard Octave version so it should perhaps not be treated as a bug. The function pownrev.m uses nthroot.m internally so it also had the same limitation. This time I would however treat it as a bug because the function pown.m does allow vectorization of the integer parameter and if that supports it the reverse function should probably also do it. So I implemented support for vectorization of the integer parameter for both nthroot.m and pownrev.m and they now pass the test.

No problems were found with the N-dimensional tests that the vector tests did not find. This is a good indication that the support for N-dimensional arrays is at least partly correct. Always good to know!

A Package for Taylor Arithmetic

In the last blog post I wrote about what was left to do with implementing support for N-dimensional arrays in the interval package. There are still some things to do but I have had, and most likely will have, some time to work on other things. Before the summer I started to work on a proof of concept implementation of Taylor arithmetic in Octave and this week I have continued to work on that. This blog post will be about that.

A Short Introduction to Taylor Arithmetic

Taylor arithmetic is a way to calculate with truncated Taylor expansions of functions. The main benefit is that it can be used to calculate derivatives of arbitrary order.

Taylor expansion or Taylor series (I will use these words interchangeably) are well known and from Wikipedia we have: The Taylor series of real or complex valued function $f(x)$ that is infinitely differentiable at a real or complex number $a$ is the power series
$$
f(a) + \frac{f'(a)}{1!}(x-a) + \frac{f''(a)}{2!}(x-a)^2 + \frac{f'''(a)}{3!}(x-a)^3 + ....
$$
From the definition it is clear that if we happen to know the coefficients of the Taylor series of $f$ at the point $a$ we can also calculate all derivatives of $f$ at that point by simply multiplying a coefficient with the corresponding factorial.

The simplest example of Taylor arithmetic is addition of two Taylor series. If $f$ has the Taylor series $\sum_{n=0}^\infty (f)_n (x-a)^n$ and $g$ the Taylor series $\sum_{n=0}^\infty (g)_n (x-a)^n$ then $f + g$ will have the Taylor series
$$
\sum_{n=0}^\infty (f + g)_n (x-a)^n = \sum_{n=0}^\infty ((f)_n + (g)_n)(x-a)^n$
$$
If we instead consider the product, $fg$, we get
$$
\sum_{n=0}^\infty (fg)_n (x-a)^n = \sum_{n=0}^\infty \left(\sum_{i=0}^n (f)_n(g)_n\right)(x-a)^n.
$$

With a bit of work you can find similar formulas for other standard functions. For example the coefficients, $(e^f)_n$, of the Taylor expansion of $\exp(f)$ is given by $(e^f)_0 = e^{(f)_0}$ and for $n > 0$
$$
(e^f)_n = \frac{1}{n}\sum_{i=0}^{n-1}(k-j)(e^f)_i(f)_{n-i}.
$$

When doing the computations on a computer we consider truncated Taylor series, we choose an order and keep only coefficients up to that order. There is also nothing that stops us from using intervals as coefficients, this allows us to get rigorous enclosures of derivatives of functions.

For a more complete introduction to Taylor arithmetic in conjunction with interval arithmetic see [1] which was my first encounter to it. For another implementation of it in code take a look at [2].

Current Implementation Status

As mentioned in the last post my repository can be found here

When I started to write on the package, before summer, my main goal was to get something working quickly. Thus I implemented the basic functions needed to do some kind of Taylor arithmetic, a constructor, some help functions and a few functions like $\exp$ and $\sin$.

This last week I have focused on implementing the basic utility functions, for example $size$, and rewriting the constructor. In the process I think I have broken the arithmetic functions, I will fix them later.

You can at least create and display Taylor expansions now. For example creating a variable $x$ with value 5 of order 3

> x = taylor (infsupdec (5), 3)
x = [5]_com + [1]_com X + [0]_com X^2 + [0]_com X^3

or a matrix with 4 variables of order 2

> X = taylor (infsupdec ([1, 2; 3, 4]), 2)
X = 2×2 Taylor matrix of order 2

ans(:,1) =

   [1]_com + [1]_com X + [0]_com X^2
   [3]_com + [1]_com X + [0]_com X^2

ans(:,2) =

   [2]_com + [1]_com X + [0]_com X^2
   [4]_com + [1]_com X + [0]_com X^2

If you want you can create a Taylor expansion with explicitly given coefficients you can do that as well

> f = taylor (infsupdec ([1; -2; 3, -4))
f = [1]_com + [-2]_com X + [3]_com X^2 + [-4]_com X^3

This would represent a function $f$ with $f(a) = 1$, $f'(a) = -2$, $f''(a) = 3 \cdot 2! = 6$ and $f'''(a) = -4 \cdot 3! = -24$.

Creating a Package

My goal is to create a full package for Taylor arithmetic along with some functions making use of it. The most important step is of course to create a working implementation but there are other things to consider as well. I have a few things I have not completely understood about it. Depending on how much time I have next week I will try to read a bit more about it probably ask some questions on the mailing list. Here are at least some of the things I have been thinking about

Mercurial vs Git?

I have understood that most of the Octave forge packages uses Mercurial for version control. I was not familiar with Mercurial before so the natural choice for me was to use Git. Now I feel I could switch to Mercurial if needed but I would like to know the potential benefits better, I'm still new to Mercurial so I don't have the full picture. One benefit is of course that it is easier if most  packages use the same system, but other than that?

How much work is it?

If I were to manage a package for Taylor arithmetic how much work is it? This summer I have been working full time with Octave so I have had lots of time but this will of course not always be the case. I know it takes time if I want to continue to improve the package, but how much, and what, continuous work is there?

What is needed besides the implementation?

From what I have understood there are a couple of things that should be included in a package besides the actual m-files. For example a Makefile for creating the release, an INDEX-file and a CITATION-file. I should probably also include some kind of documentation, especially since Taylor arithmetic is not that well known. Is there anything else I need to think about?

What is the process to get a package approved?

If I were to apply (whatever that means) for the package to go to Octave forge what is the process for that? What is required before it can be approved and what is required after it is approved?


[1] W. Tucker, Validated Numerics, Princeton University Press, 2011.
[2] F. Blomquist, W. Hofschuster, W. Krämer, Real and complex taylor arithmetic in C-XSC, Preprint 2005/4, Bergische Universität Wuppertal.

Ahead of the Timeline

One of my first posts on this blog was a timeline for my work during the project. Predicting the amount of time something takes is always hard to do. Often time you tend to underestimate the complexity of parts of the work. This time however I overestimated the time the work would take.

If my timeline would have been correct I would have just started to work on Folding Functions (or reductions as they are often called). Instead I have completed the work on them and also for functions regarding plotting. In addition I have started to work on the documentation for the package as well as checking everything an extra time.

In this blog post I will go through what I have done this week, what I think is left to do and a little bit about what I might do if I complete the work on N-dimensional arrays in good time.

This Week

The Dot Function

The $dot$-function was the last function left to implement support for N-dimensional arrays in. It is very similar to the $sum$-function so I already had an idea of how to do it. As with  $sum$ I moved most of the handling of the vectorization from the m-files to the oct-file, the main reason being improved performance.

The $dot$-functions for intervals is actually a bit different from the standard one. First of all it supports vectorization which the standard one does not

> dot ([1, 2, 3; 4, 5, 6], 5)
error: dot: size of X and Y must match
> dot (infsupdec ([1, 2, 3; 4, 5, 6], 5)
ans = 1x3 interval vector

  [25]_com   [35]_com   [45]_com

It also treats empty arrays a little different, see bug #51333,

> dot ([], [])
ans = [](1x0)
> dot (infsupdec ([]), [])
ans = [0]_com

Package Documentation

I have done the minimal required changes to the documentation. That is I moved support for N-dimensional arrays from Limitation to Features and added some simple examples on how to create N-dimensional arrays.

Searching for Misses

During the work I have tried to update the documentation for all functions to account for the support of N-dimensional arrays and I have also tried to update some of the comments for the code. But as always, especially when working with a lot of files, you miss things, both in the documentation and old comments.

I did a quick grep for the words "matrix" and "matrices" since they are candidates for being changed to "array". Doing this I found 35 files where I missed things. It was mainly minor things, comments using the "matrix" which I now changed to "array", but also some documentation which I had forgotten to update.

What is Left?

Package Documentation - Examples

As mentioned above I have done the minimal required changes to the documentation. It would be very nice to add some more interesting example using N-dimensional arrays of intervals in a useful way. Ironically I have not been able to come up with an interesting example but I will continue to think about it. If you have an example that you think would be interesting and want to share, please let me know!

Coding Style

As I mentioned in one of the first blog posts, the coding style for the interval package was not following the standard for Octave. During my work I have adapted all files I have worked with to the coding standard for Octave. A lot of the files I have not needed to do any changes to, so they are still using the old style. It would probably be a good idea to update them as well.

Testing - ITF1788

The interval framwork libary developed by Oliver is used to test the correctness of many of the functions in the package. At the moment it tests evaluation of scalars but in principle it should be no problem to use it for testing vectorization or even broadcasting. Oliver has already started to work on this.

After N-dimensional arrays?

If I continue at this pace I will finish the work on N-dimensional arrays before the time of the project is over. Of course the things that are left might take longer than expected, they usually do, but there is a chance that I will have time left after everything is done. So what should I do then? There are more thing that can be done on the interval package, for example adding more examples to the documentation, however I think I would like to start working on a new package for Taylor arithmetics.

Before GSoC I started to implement a proof of concept for Taylor arithmetics in Ocatve which can be found here. I would then start to work on implementing a proper version of it, where I would actually make use of N-dimensional interval arrays. If I want to create a package for this I would also need to learn a lot of other things, one of them being how to manage a package on octave forge.

At the moment I will try to finish my work on N-dimensional arrays. Then I can discuss it with Oliver and see what he thinks about it.

Set inversion with fsolve

This week my work have mainly been focused on the interval version of fsolve. I was not sure if and how this could make use of N-dimensional arrays and to find that out I had to understand the function. In the end it turned out that the only generalization that could be done were trivial and required very few changes. However I did find some other problems with the functions that I have been able to fix. Connected to fsolve are the functions ctc_intersect and ctc_union. The also needed only minor changes to allow for N-dimensional input. I will start by giving an introduction to fsolve, ctc_union and ctc_intersect and then I will mention the changes I have done to them.

Introduction to fsolve

The standard version of fsolve in Octave is used to solve systems of nonlinear equations. That is, given a functions $f$ and a starting point $x_0$ it returns a value $x$ such that $f(x)$ is close to zero. The interval version of fsolve does much more than this. It is used to enclose the preimage of a set $Y$ under $f$. Given a domain $X$, a set $Y$ and a function $f$ it returns an enclosure of the set
$$
f^{-1}(Y) = \{x \in X: f(x) \in Y\}.
$$
By letting $Y = 0$ we get similar functionality as the standard fsolve, with the difference that the output is an enclosure of all zeros to the function (compared to one point for which $f$ returns close to zero).

Example: The Unit Circle

Consider the function $f(x, y) = \sqrt{x^2 + y^2} - 1$ which is zero exactly on the unit circle. Plugging this in to the standard fsolve we get with $(0.5, 0.5)$ as a starting guess

> x = fsolve (@(x) f(x(1), x(2)), [0.5, 0.5])
x = 0.70711 0.70711

which indeed is close to a zero. But we get no information about other zeros.

Using the interval version of fsolve with $X = [-3, 3] \times [-3, 3]$ as starting domain we get

> [x paving] = fsolve (f, infsup ([-3, -3], [3, 3]));
> x
x ⊂ 2×1 interval vector

     [-1.002, +1.002]
   [-1.0079, +1.0079]

Plotting the paving we get the picture

which indeed is a good enclosure of the unit circle.

How it works

In its simplest form fsolve uses a simple bisection scheme to find the enclosure. Using interval methods we can find enclosure to images of sets. Given a set $X_0 \subset X$ we have three different possibilities

• $f(X_0) \subset Y$ in which case we add $X_0$ to the paving
• $f(X_0) \cap Y = \emptyset$ in which case we discard $X_0$
• Otherwise we bisect $X_0$ and continue on the parts

By setting a tolerance of when to stop bisecting boxes we get the algorithm to terminate in a finite number of steps.

Contractors

Using bisection is not always very efficient, especially when the domain has many dimensions. One way to speed up the convergence is with what's called contractors. In short a contractor is a function that can take the set $X_0$ and returns a set $X_0' \subset X_0$ with the property that $f(X_0 \setminus X_0') \cap Y = \emptyset$. It's a way of making $X_0$ smaller without having to bisect it that many times.

When you construct a contractor you use the reverse operations definer on intervals. I will not go into how this works, if you are interested you can find more information in the package documentation [1] and in these youtube videos about Set Inversion Via Interval Analysis (SIVIA) [2].

The functions ctc_union and ctc_intersect are used to combine contractors on sets into contractors on unions or intersections of these sets.

Generalization to N-dimensional arrays

How can fsolve be generalized to N-dimensional arrays? The only natural thing to do is to allow for the input and output of $f$ to be N-dimensional arrays. This also is no problem to do. While you mathematically probably would say that fsolve is used to do set inversion for functions $f: \mathbb{R}^n \to \mathbb{R}^m$ it can of course also be used for example on functions $f: \mathbb{R}^{n_1}\times \mathbb{R}^{n_2} \to \mathbb{R}^{m_1}\times \mathbb{R}^{m_2}$.

This is however a bit different when using vectorization. When not using vectorization (and not using contractions) fsolve expects that the functions takes one argument which is an array with each element corresponding to a variable. If vectorization is used it instead assumes that the functions takes one argument for each variable. Every argument is then given as a vector with each element corresponding to one value of the variable for which to compute the function. Here we have no use of N-dimensional arrays.

Modifications

The only change in functionality that I have done to the functions is to allow for N-dimensional arrays as input and output when vectorization is not used. This required only minor changes, essentially changing expressions like
max (max (wid (interval)))
to
max (wid (interval)(:))
It was also enough to do these changes in ctc_union and ctc_intersect to have these support N-dimensional arrays.

I have made no functional changes when vectorization is used. I have however made an optimization in the construction of the arguments to the function. The arguments are stored in an array but before being given to the function they need to be split up into the different variables. This is done by creating a cell array with each element being a vector with the values of one of the variables. Previously the construction of this cell array was very inefficient, it split the interval into its lower and upper part and then called the constructor to create an interval again. Now it copies the intervals into the cell without having to call the constructor. This actually seems have been quite a big improvement, using the old version the example with the unit circle from above took around 0.129 seconds and with the new version it takes about 0.092 seconds. This is of course only one benchmark, but a speed up of about 40% for this test is promising!

Lastly I noticed a problem in the example used in the documentation of the function. The function used is

# Solve x1 ^ 2 + x2 ^ 2 = 1 for -3 ≤ x1, x2 ≤ 3 again,
# but now contractions speed up the algorithm.
function [fval, cx1, cx2] = f (y, x1, x2)
  # Forward evaluation
  x1_sqr = x1 .^ 2;
  x2_sqr = x2 .^ 2;
  fval = hypot (x1, x2);
  # Reverse evaluation and contraction
  y = intersect (y, fval);
  # Contract the squares
  x1_sqr = intersect (x1_sqr, y - x2_sqr);
  x2_sqr = intersect (x2_sqr, y - x1_sqr);
  # Contract the parameters
  cx1 = sqrrev (x1_sqr, x1);
  cx2 = sqrrev (x2_sqr, x2);
endfunction

Do you see the problem? I think it took me more than a day to realize that the problems I was having was not because of a bug in fsolve but because this function computes the wrong thing. The function is supposed to be $f(x_1, x_2) = x_1^2 + x_2^2$ but when calculating the value it calls hypot which is given by $hypot(x_1, x_2) = \sqrt{x_1^2 + x_2^2}$. For $f(x_1, x_2) = 1$, which is used in the example, it gives the same result, but otherwise it will of course not work.

[1] https://octave.sourceforge.io/interval/package_doc/Parameter-Estimation.html#Parameter-Estimation
[2] https://www.youtube.com/watch?v=kxYh2cXHdNQ

One Month In

Now one month of GSoC has passed and so far everything has gone much better than I expected! According to my timeline this week would have been the first of two were I work on vectorization. Instead I have already mostly finished the vectorization and have started to work on other things. In this blog post I'll give a summary of what work I have completed and what I have left to do. I'll structure it according to where the functions are listed in the $INDEX$-file [1]. The number after the heading is the number of functions in that category.

Since this will mainly be a list of which files have been modified and which are left to do this might not be very interesting if you are not familiar with the structure of the interval package.

Interval constant (3)

All of these have been modified to support N-dimensional arrays.

Interval constructor (5)

All of these have been modified to support N-dimensional arrays.

Interval function (most with tightest accuracy) (63)

Almost all of these functions worked out of the box! At least after the API functions to the MPFR and crlibm libraries were fixed, they are further down in the list.

The only function that did not work immediately were $linspace$. Even though this function could be generalized to N-dimensinal arrays the standard Octave function only works for matrices (I think the Matlab version only allows scalars). This means that adding support for N-dimensional vectors for the interval version is not a priority. I might do it later on but it is not necessary.

Interval matrix operation (16)

Most of the matrix functions does not make sense for N-dimensional arrays. For example matrix multiplication and matrix inversion only makes sense for matrices. However all of the reduction functions are also here, they include $dot$, $prod$, $sum$, $sumabs$ and $sumsq$.

At the moment I have implemented support for N-dimensional arrays for $sum$, $sumabs$ and $prod$. The functions $dot$ and $sumsq$ are not ready, I'm waiting to see what happens with bug #51333 [2] before I continue with that work. Depending on the bug I might also have to modify the behaviour of $sum$, $sumabs$ and $prod$ slightly.

Interval comparison (19)

All of these have been modified to support N-dimensional arrays.

Set operation (7)

All of these functions have been modified to support N-dimensional arrays except one, $mince$. The function $mince$ is an interval version of $linspace$ and reasoning here is the same as that for $linspace$ above.

Interval reverse operation (12)

Like the interval functions above, all of the functions worked out of the box!

Interval numeric function (11)

Also these functions worked out of the box, with some small modifications to the documentation for some of them.

Interval input and output (9)

Here there are some functions which require some comments, the ones I do not comment about have all gotten support for N-dimensional arrays.

$interval\_bitpack$
I think that this function does not make sense to generalize to N-dimensions. It could perhaps take an N-dimensional arrays as input, but it will always return a row vector.  I have left it as it is for now at least.

$disp$ and $display$
These are functions that might be subject to change later on. At the moment it prints N-dimensional arrays of intervals in the same way Octave does for normal arrays. It's however not clear how to handle the $\subset$ symbol and we might decide to change it.

Interval solver or optimizer (5)

The functions $gauss$ and $polyval$ are not generalizable to N-dimensional vectors. I don't think that $fzero$ can be generalized either, for it to work the functions must be real-valued.

The function $fsolve$ can perhaps be modified to support N-dimensional vectors. It uses the SIVIA algorithm [3] and I have to dive deeper into how it works to see if it can be done.

For $fminsearch$ nothing needed to be done, it worked for N-dimensional arrays directly.

Interval contractor arithmetic (2)

Both of these functions are used together with $fsolve$ so they also depend on if SIVIA can be generalized or not.

Verified solver or optimizer (6)

All of these functions work on matrices and cannot be generalized.

Utility function (29)

All of these for which it made sense have been modified to support N-dimensional arrays. Some of them only works for matrices, these are $ctranspose$, $diag$, $transpose$, $tril$ and $triu$. I have left them as they were, though I fixed a bug in $diag$.

API function to the MPFR and crlibm libraries (8)

These are the functions that in general required most work. The ones I have added full support for N-dimensional arrays in are $crlibm\_function$, $mpfr\_function\_d$ and $mpfr\_vector\_sum\_d$. Some of them cannot be generalized, these are $mpfr\_matrix\_mul_d$, $mpfr\_matrix\_sqr\_d$ and $mpfr\_to\_string\_d$. The functions $mpfr\_linspace\_d$ and $mpfr\_vector\_dot\_d$ are related to what I mentioned above for $linspace$ and $dot$.

Summary

So summing up the functions that still require some work are

• Functions related to $fsolve$
• The functions $dot$ and $sumsq$
• The functions $linspace$ and $mince$

Especially the functions related to $fsolve$ might take some time to handle. My goal is to dive deeper into this next week.

Apart from this there are also some more things that needs to be considered. The documentation for the package will need to be updated. This includes adding some examples which make use of the new functionality.

The interval package also did not follow the coding style for Octave. All the functions which I have made changes to have been updated with the correct coding style, but many of the functions that worked out of the box still use the old style. It might be that we want to unify the coding standard for all files before the next release.

[1] The $INDEX$ file https://sourceforge.net/u/urathai/octave/ci/default/tree/INDEX
[2] Bug #51333 https://savannah.gnu.org/bugs/index.php?51333
[3] The SIVIA algorithm https://www.youtube.com/watch?v=kxYh2cXHdNQ