Failed to install Python package - update request

[parshakova]

I cannot install and use the Python wrapper for HODLR.
Could you please let me know if you are going to update the package?

[shyams2]

Hi @parshakova

I had originally worked on the Python wrapper when it was first written. I can possibly take a look at it again over this weekend or the next.

[parshakova]

Thank you very much, that would be super helpful!

In the install.sh file, link http://bitbucket.org/eigen/eigen/get/3.3.7.tar.bz2 no longer exists.
So I just stored https://gitlab.com/libeigen/eigen in the /deps/eigen folder

[parshakova]

@shyams2 have you had time to look at the Python wrapper ?
Thank you!

[shyams2]

@parshakova I checked it out. Seems like I was able to build it and run it. However, it does seem like you'll have to pull the latest version of pybind11. The earlier state that's frozen on the repository seems to fail. That is:

cd HODLR/python
rm -rf pybind11
git clone https://github.com/pybind/pybind11.git
python setup.py install

If you do have the dependencies and the environment variables setup, I think it should install fine. Can you try that out?

[parshakova]

I get the following error

/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/ForwardDeclarations.h/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/ForwardDeclarations.h::112112::4646::  error: error: no type named 'conditional_t' in namespace 'std'no type named 'conditional_t' in namespace 'std'

         typename StrideType = typename std::conditional_t<PlainObjectType::IsVectorAtCompileTime,InnerStride<1>,OuterStride<> > > class Ref;
                               ~~~~~~~~~~~~~~^~~~~~~~~~~~~
         typename StrideType = typename std::conditional_t<PlainObjectType::IsVectorAtCompileTime,InnerStride<1>,OuterStride<> > > class Ref;
                               ~~~~~~~~~~~~~~^~~~~~~~~~~~~
/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/ForwardDeclarations.h:112:59: error: /Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/ForwardDeclarations.h:expected ',' or '>' in template-parameter-list112:
59: error: expected ',' or '>' in template-parameter-list
         typename StrideType = typename std::conditional_t<PlainObjectType::IsVectorAtCompileTime,InnerStride<1>,OuterStride<> > > class Ref;
                                                          ^
/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/ForwardDeclarations.h:112:         typename StrideType = typename std::conditional_t<PlainObjectType::IsVectorAtCompileTime,InnerStride<1>,OuterStride<> > > class Ref;98
:                                                           ^
error: expected unqualified-id
/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/ForwardDeclarations.h:112:98: error: expected unqualified-id
         typename StrideType = typename std::conditional_t<PlainObjectType::IsVectorAtCompileTime,InnerStride<1>,OuterStride<> > > class Ref;
                                                                                                 ^
         typename StrideType = typename std::conditional_t<PlainObjectType::IsVectorAtCompileTime,InnerStride<1>,OuterStride<> > > class Ref;
                                                                                                 ^
In file included from /Users/tetiana/Documents/Hierarchical_matrices/HODLR/python/wrapper/pyhodlrlib.cpp:1:
In file included from /Users/tetiana/Documents/Hierarchical_matrices/HODLR/include/HODLR_Matrix.hpp:8:
In file included from /Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/Dense:1:
In file included from /Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/Core:172:
/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/XprHelper.h:105:In file included from 16/Users/tetiana/Documents/Hierarchical_matrices/HODLR/src/HODLR_Matrix.cpp:: 1:
In file included from error: /Users/tetiana/Documents/Hierarchical_matrices/HODLR/include/HODLR_Matrix.hpp:8no template named 'conditional_t' in namespace 'std':
In file included from
/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/Dense:1:
In file included from /Users/parsh  typedef std::conditional_t<(sizeof(I1)<sizeof(I2)), I2, I1> type;akova.tanya/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/Core
:          ~~~~~^
172:
/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/XprHelper.h:105:16: error: no template named 'conditional_t' in namespace 'std'
  typedef std::conditional_t<(sizeof(I1)<sizeof(I2)), I2, I1> type;
          ~~~~~^
/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/XprHelper.h:195:3: warning: use of this statement in a constexpr function is a C++14 extension [-Wc++14-extensions]
  if((ArrayBytes % AlignmentBytes) == 0) {
  ^
/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/XprHelper.h:200:5: warning: multiple return statements in constexpr function is a C++14 extension [-Wc++14-extensions]
    return 0;
    ^
/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/XprHelper.h:196:5: note: previous return statement is here
    return AlignmentBytes;
    ^
/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/XprHelper.h/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/XprHelper.h::198195::53::  note: warning: previous return statement is hereuse of this statement in a constexpr function is a C++14 extension [-Wc++14-extensions]

  if((ArrayBytes % AlignmentBytes) == 0) {
  ^    return compute_default_alignment_helper(ArrayBytes, AlignmentBytes/2);

    ^
/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/XprHelper.h:200:5: warning: multiple return statements in constexpr function is a C++14 extension [-Wc++14-extensions]
    return 0;
    ^
/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/XprHelper.h:196:5: note: previous return statement is here
    return AlignmentBytes;
    ^
/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/XprHelper.h:198:5: note: previous return statement is here
    return compute_default_alignment_helper(ArrayBytes, AlignmentBytes/2);
    ^
/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/XprHelper.h:242:12: warning: variable declaration in a constexpr function is a C++14 extension [-Wc++14-extensions]
  unsigned row_major_bit = Options&RowMajor ? RowMajorBit : 0;
           ^
/Users/tetiana/Documents/Hierarchical_matrices/HODLR/deps/eigen/Eigen/src/Core/util/XprHelper.h:242:12: warning: variable declaration in a constexpr function is a C++14 extension [-Wc++14-extensions]
  unsigned row_major_bit = Options&RowMajor ? RowMajorBit : 0;
           ^
fatal error: fatal error: too many errors emitted, stopping now [-ferror-limit=]too many errors emitted, stopping now [-ferror-limit=]

10 warnings and 20 errors generated.
make[2]: *** [CMakeFiles/pyhodlrlib.dir/Users/tetiana/Documents/Hierarchical_matrices/HODLR/src/HODLR_Matrix.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
10 warnings and 20 errors generated.
make[2]: *** [CMakeFiles/pyhodlrlib.dir/wrapper/pyhodlrlib.cpp.o] Error 1
make[1]: *** [CMakeFiles/pyhodlrlib.dir/all] Error 2
make: *** [all] Error 2
Traceback (most recent call last):
  File "/Users/tetiana/Documents/Hierarchical_matrices/HODLR/python/setup.py", line 69, in <module>
    setup(
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
    self.run_command(cmd)
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/command/install.py", line 74, in run
    self.do_egg_install()
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/command/install.py", line 123, in do_egg_install
    self.run_command('bdist_egg')
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/command/bdist_egg.py", line 165, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command
    self.run_command(cmdname)
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/_distutils/command/install_lib.py", line 112, in build
    self.run_command('build_ext')
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
    cmd_obj.run()
  File "/Users/tetiana/Documents/Hierarchical_matrices/HODLR/python/setup.py", line 35, in run
    self.build_extension(ext)
  File "/Users/tetiana/Documents/Hierarchical_matrices/HODLR/python/setup.py", line 65, in build_extension
    subprocess.check_call(['cmake', '--build', '.'] + build_args,
  File "/Users/tetiana/opt/anaconda3/envs/routing/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--config', 'Release', '--', '-j2']' returned non-zero exit status 2.

[shyams2]

@parshakova can you give me details about your build environment. Version of CMake / g++ you're using and the version of Eigen that you're utilizing.

It seems like the compiler that you're using doesn't seem to be enabled for C++11 features. Atleast that's what I'm inferring given that you see no type named 'conditional_t' in namespace 'std'

Also, have the environment variables for HODLR_PATH and EIGEN_PATH been assigned?

[parshakova]

Thank you for fast replies. Environment variables are defined

$ echo $HODLR_PATH
/Documents/Hierarchical_matrices/HODLR
$ echo $EIGEN_PATH
/Documents/Hierarchical_matrices/HODLR/deps/eigen/
$ cmake --version
cmake version 3.26.3
$ g++ --version
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: x86_64-apple-darwin22.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

I got eigen from here https://gitlab.com/libeigen/eigen master branch

Thank you!

[shyams2]

Hmm... I'm not too familiar with the Apple Clang compiler. Let's try something out - can you see if you can build the C++ examples:

cd HODLR
mkdir build
cd build
cmake ..
make all

If that fails, can you maybe try installing the gcc compiler and see if it works with that?

[parshakova]

on Linux platform with

eigen 3.4
$ g++ --version
g++ (GCC) 12.1.0
$ cmake --version
cmake version 3.12.4

I get the eigen error during make all

/home/users/tetianap/HODLR/include/HODLR_Node.hpp:37:21:   required from here
/home/users/tetianap/HODLR/deps/eigen/Eigen/src/Cholesky/LLT.h:403:85: error: no type named ‘AdjointReturnType’ in ‘class Eigen::Matrix<double, -1, -1>’
  403 |   typedef const TriangularView<const typename MatrixType::AdjointReturnType, Upper> MatrixU;
      |                                                                                     ^~~~~~~
In file included from /home/users/tetianap/HODLR/deps/eigen/Eigen/Core:179,
                 from /home/users/tetianap/HODLR/deps/eigen/Eigen/Dense:1,
                 from /home/users/tetianap/HODLR/include/HODLR_Matrix.hpp:8,
                 from /home/users/tetianap/HODLR/examples/example_conducting_sphere_plate.cpp:1:
/home/users/tetianap/HODLR/deps/eigen/Eigen/src/Core/MathFunctions.h: In instantiation of ‘bool Eigen::numext::isnan(const T&) [with T = float]’:
/home/users/tetianap/HODLR/deps/eigen/Eigen/src/Core/arch/Default/BFloat16.h:351:40:   required from here
/home/users/tetianap/HODLR/deps/eigen/Eigen/src/Core/MathFunctions.h:1302:97: error: no matching function for call to ‘isnan_impl(const/home/users/tetianap/HODLR/deps/eigen/Eigen/src/Core/MatrixBase.h:97:61: error: ‘constexpr int Eigen::internal::max_size_prefer_dynamic
(A, B) [with A = Eigen::DenseBase<Eigen::Matrix<double, -1, -1> >::<unnamed enum>; B = Eigen::DenseBase<Eigen::Matrix<double, -1, -1> >
::<unnamed enum>]’ called in a constant expression
   97 |     typedef Matrix<Scalar, internal::max_size_prefer_dynamic(RowsAtCompileTime, ColsAtCompileTime),
      |                            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/users/tetianap/HODLR/deps/eigen/Eigen/Core:170,
                 from /home/users/tetianap/HODLR/deps/eigen/Eigen/Dense:1,
                 from /home/users/tetianap/HODLR/include/HODLR_Matrix.hpp:8,
                 from /home/users/tetianap/HODLR/examples/example_conducting_sphere_plate.cpp:1:
/home/users/tetianap/HODLR/deps/eigen/Eigen/src/Core/util/ForwardDeclarations.h:55:35: note: in template argument for type ‘int’

[shyams2]

@parshakova , I tried working with the development version of Eigen from the master branch of https://gitlab.com/libeigen/eigen as well. Seems like that gives me an error in compilation as well. Maybe some development changes are messing things up. Can you try using the 3.4 release version and see if that works? That is replace the current eigen folder with

 wget https://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.gz
 tar xzf eigen-3.4.0.tar.gz
 mv eigen-3.4.0 eigen

[parshakova]

It works now on Linux !
Thank you!

could you please let me know how to check the storage for matrix in HODLR format?

[shyams2]

Not sure I completely follow. Do you mean something like seeing memory storage for a HODLR matrix?

[parshakova]

Yes, to see the compression factor as opposed to original matrix storage $mn$

[shyams2]

Don't believe there was a direct function that returned the total memory consumption. However, you can get the size at each leaf level. One easy way to see it is just using plotTree that'll show the rank at each level so you can infer the memory usage