Hi all, I'm trying to verify the ngspice build with the tests that are on the

tests

directory. The regression ones are fine but the rest of them (bsim3, jfet, mos6) fails or passes with a lot of warnings. I've used the compiled ngspice to run simulations of sky130 and ihp pdks and it works fine, it gives the expected result. The evaluation system runs Ubuntu 24.04 over a docker container on windows. This are the summarized results for the non-regression tests:

make -C tests/bsim1        check # failed 1/1
make -C tests/bsim2        check # failed 1/1
make -C tests/bsim3        check # Pass, most of checks out of the ranges
make -C tests/bsim3soidd   check # failed 2/5
make -C tests/bsim3soifd   check # failed 3/5
make -C tests/bsim3soipd   check # failed 2/5
make -C tests/bsim4        check # Pass, most of checks out of the ranges
make -C tests/bsimbulk     check # No rule to make target "check"
make -C tests/bsimsoi      check # Pass, most of checks out of the ranges
make -C tests/filters      check # failed 1/1
make -C tests/general      check # failed 8/8
make -C tests/hfet         check # failed 2/2
make -C tests/hicum2       check # Pass, most of checks out of the ranges
make -C tests/hisim        check # Pass, most of checks out of the ranges
make -C tests/hisimhv1     check # Pass, most of checks out of the ranges
make -C tests/hisimhv2     check # Pass, most of checks out of the ranges
make -C tests/jfet         check # failed 1/1
make -C tests/mes          check # failed 1/1
make -C tests/mesa         check # failed 7/9
make -C tests/mos6         check # failed 2/2
make -C tests/polezero     check # failed 6/6
make -C tests/resistance   check # failed 3/3
make -C tests/sensitivity  check # failed 1/1 
make -C tests/transient    check # failed 1/1 
make -C tests/transmission check # failed 4/6 
make -C tests/vbic         check # failed 4/6 
make -C tests/xspice       check # Pass

It seems that the tests with

****** Checking test (ngspice):

always passes, despite the errors. Some of the maximum relative error: • standard: 99% • m: 4.88% • Flip_N: 93.9% On the regression tests some of them writes warnings and errors, but all pass. lib-processing: • On `scope-3.cir`:

Warning: Nesting of subcircuits with parameters is only marginally supported!

misc: • On `asrc-tc-1.cir`: Writes some warnings and errors.

Warning from checkvalid: vector v2#branch is not available or has zero length.
  Error: RHS "i(v2)" invalid
  Error: no such vector v(3)
  Error: Can't evaluate "v(3)"
  Warning from checkvalid: vector iac2 is not available or has zero length.
  Error: RHS "iac2/iac2_gold - 1" invalid
  Warning from checkvalid: vector vac3 is not available or has zero length.
  Error: RHS "vac3/vac3_gold - 1" invalid
  Warning from checkvalid: vector err3 is not available or has zero length.

• On `asrc-tc-2.cir`: Writes some of warnings and errors.

Warning from checkvalid: vector v2#branch is not available or has zero length.
  Error: RHS "i(v2)" invalid
  Warning from checkvalid: vector v3#branch is not available or has zero length.
  Error: RHS "i(v3)" invalid
  Warning from checkvalid: vector iac2 is not available or has zero length.
  Error: RHS "iac2/iac_gold - 1" invalid
  Warning from checkvalid: vector iac3 is not available or has zero length.
  Error: RHS "iac3/iac_gold - 1" invalid
  Warning from checkvalid: vector err3 is not available or has zero length.

pipe: • on `alter-1.cmd`: Seems to require X-server, despite of this the test doesn't increment the fail_count variable so I think this is fine Does this indicates that the ngspice binary is wrong? Some of the tests that are not on the

regression/

directory have more than 10 years without modifications. Maybe they are not valid on ngspice-42. Since all regression tests pass. Should I care about the warnings and errors on

misc

,

lib-processing

and

pipe

? Thanks for the help @Mitch Bailey The tip of using

make check

on the build directory worked. Is Holger Vogt on this slack?