Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test error #36

Open
Cody-G opened this issue Feb 21, 2017 · 16 comments
Open

Test error #36

Cody-G opened this issue Feb 21, 2017 · 16 comments

Comments

@Cody-G
Copy link
Contributor

Cody-G commented Feb 21, 2017

I'm getting this test error in Julia 0.5. I also tried starting from a fresh empty package directory and still get the same error.

ERROR: LoadError: LoadError: BoundsError: attempt to access 6-element UnitRange{Int64} at index [-2:13]
 in throw_boundserror(::UnitRange{Int64}, ::Tuple{UnitRange{Int64}}) at ./abstractarray.jl:355
 in checkbounds at ./abstractarray.jl:284 [inlined]
 in getindex at ./range.jl:504 [inlined]
 in unsafe_reindex at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatchCommon.jl:437 [inlined]
 in extraunsafe_view at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatchCommon.jl:433 [inlined]
 in safe_get!(::Array{Float64,2}, ::SubArray{Float64,2,Array{Float64,2},Tuple{UnitRange{Int64},UnitRange{Int64}},false}, ::Tuple{UnitRange{Int64},UnitRange{Int64}}, ::Float64) at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatchCommon.jl:387
 in #mismatch!#33(::Symbol, ::Function, ::CenterIndexedArrays.CenterIndexedArray{RegisterCore.NumDenom{Float64},2,Array{RegisterCore.NumDenom{Float64},2}}, ::RegisterMismatch.CMStorage{Float64,2}, ::SubArray{Float64,2,Array{Float64,2},Tuple{UnitRange{Int64},UnitRange{Int64}},false}) at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatch.jl:136
 in (::RegisterMismatch.#kw##mismatch!)(::Array{Any,1}, ::RegisterMismatch.#mismatch!, ::CenterIndexedArrays.CenterIndexedArray{RegisterCore.NumDenom{Float64},2,Array{RegisterCore.NumDenom{Float64},2}}, ::RegisterMismatch.CMStorage{Float64,2}, ::SubArray{Float64,2,Array{Float64,2},Tuple{UnitRange{Int64},UnitRange{Int64}},false}) at ./<missing>:0
 in #mismatch_apertures!#35(::Symbol, ::Function, ::Array{CenterIndexedArrays.CenterIndexedArray{RegisterCore.NumDenom{Float64},2,Array{RegisterCore.NumDenom{Float64},2}},2}, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Tuple{Float64,Float64},2}, ::RegisterMismatch.CMStorage{Float64,2}) at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatch.jl:230
 in (::RegisterMismatch.#kw##mismatch_apertures!)(::Array{Any,1}, ::RegisterMismatch.#mismatch_apertures!, ::Array{CenterIndexedArrays.CenterIndexedArray{RegisterCore.NumDenom{Float64},2,Array{RegisterCore.NumDenom{Float64},2}},2}, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Tuple{Float64,Float64},2}, ::RegisterMismatch.CMStorage{Float64,2}) at ./<missing>:0
 in #mismatch_apertures#34(::Symbol, ::UInt32, ::Array{Any,1}, ::Function, ::Type{Float64}, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Tuple{Float64,Float64},2}, ::Tuple{Float64,Float64}, ::Tuple{Int64,Int64}) at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatch.jl:207
 in (::RegisterMismatch.#kw##mismatch_apertures)(::Array{Any,1}, ::RegisterMismatch.#mismatch_apertures, ::Type{Float64}, ::Array{Float64,2}, ::Array{Float64,2}, ::Array{Tuple{Float64,Float64},2}, ::Tuple{Float64,Float64}, ::Tuple{Int64,Int64}) at ./<missing>:0
 in #mismatch_apertures#5(::Array{Any,1}, ::Function, ::Type{Float64}, ::Array{Float64,2}, ::Array{Float64,2}, ::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}) at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatchCommon.jl:20
 in (::RegisterMismatch.#kw##mismatch_apertures)(::Array{Any,1}, ::RegisterMismatch.#mismatch_apertures, ::Type{Float64}, ::Array{Float64,2}, ::Array{Float64,2}, ::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}) at ./<missing>:0
 in #mismatch_apertures#3(::Array{Any,1}, ::Function, ::Array{Float64,2}, ::Array{Float64,2}, ::Tuple{Int64,Int64}, ::Vararg{Tuple{Int64,Int64},N}) at /home/cody/.julia/v0.5/BlockRegistration/src/RegisterMismatchCommon.jl:13
 in (::RegisterMismatch.#kw##mismatch_apertures)(::Array{Any,1}, ::RegisterMismatch.#mismatch_apertures, ::Array{Float64,2}, ::Array{Float64,2}, ::Tuple{Int64,Int64}, ::Tuple{Int64,Int64}) at ./<missing>:0
 in include_from_node1(::String) at ./loading.jl:488
 in runtest(::String) at /home/cody/.julia/v0.5/BlockRegistration/test/runtests.jl:6
 in include_from_node1(::String) at ./loading.jl:488
 in process_options(::Base.JLOptions) at ./client.jl:262
 in _start() at ./client.jl:318
while loading /home/cody/.julia/v0.5/BlockRegistration/test/register_mismatch.jl, in expression starting on line 104
while loading /home/cody/.julia/v0.5/BlockRegistration/test/runtests.jl, in expression starting on line 17
WARNING: Forcibly interrupting busy workers
WARNING: rmprocs: process 1 not removed
=========================================================[ ERROR: BlockRegistration ]==========================================================

failed process: Process(`/home/cody/git/julia/usr/bin/julia -Cnative -J/home/cody/git/julia/usr/lib/julia/sys.so --compile=yes --depwarn=yes --check-bounds=yes --code-coverage=none --color=yes --compilecache=yes /home/cody/.julia/v0.5/BlockRegistration/test/runtests.jl`, ProcessExited(1)) [1]

===============================================================================================================================================
ERROR: BlockRegistration had test errors
 in #test#61(::Bool, ::Function, ::Array{AbstractString,1}) at ./pkg/entry.jl:740
 in (::Base.Pkg.Entry.#kw##test)(::Array{Any,1}, ::Base.Pkg.Entry.#test, ::Array{AbstractString,1}) at ./<missing>:0
 in (::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#test,Tuple{Array{AbstractString,1}}})() at ./pkg/dir.jl:31
 in cd(::Base.Pkg.Dir.##2#3{Array{Any,1},Base.Pkg.Entry.#test,Tuple{Array{AbstractString,1}}}, ::String) at ./file.jl:59
 in #cd#1(::Array{Any,1}, ::Function, ::Function, ::Array{AbstractString,1}, ::Vararg{Array{AbstractString,1},N}) at ./pkg/dir.jl:31
 in (::Base.Pkg.Dir.#kw##cd)(::Array{Any,1}, ::Base.Pkg.Dir.#cd, ::Function, ::Array{AbstractString,1}, ::Vararg{Array{AbstractString,1},N}) at ./<missing>:0
 in #test#3(::Bool, ::Function, ::String, ::Vararg{String,N}) at ./pkg/pkg.jl:258
 in test(::String, ::Vararg{String,N}) at ./pkg/pkg.jl:258
@Cody-G
Copy link
Contributor Author

Cody-G commented Feb 21, 2017

Maybe I should be running the release-0.5 branch of Julia? Currently on the tagged v0.5.0

@Cody-G
Copy link
Contributor Author

Cody-G commented Feb 21, 2017

Oh I forgot that we are using a patched Julia on cannon. I'll replicate the patch and see if that fixes it.

@Cody-G
Copy link
Contributor Author

Cody-G commented Feb 21, 2017

Hmm I'm still getting the same error when I run the latest "release-0.5" branch with the patch that we have on cannon now

@Cody-G
Copy link
Contributor Author

Cody-G commented Feb 21, 2017

Actually I'm getting the same error on cannon too, and it was also triggered by the latest Travis build

@mdhe1248
Copy link
Contributor

mdhe1248 commented Feb 21, 2017 via email

@timholy
Copy link
Member

timholy commented Feb 21, 2017

If I use the version of BlockRegistration that's installed at /usr/local/julia/julia-0.5/usr/share/julia/site/v0.5/BlockRegistration/test (which is what you use if you don't have it installed in your own private v0.5 directory), then the register_mismatch.jl test passes for me. Looks like I missed a depwarn or two in RegisterMismatchCuda, though.

@Cody-G
Copy link
Contributor Author

Cody-G commented Feb 21, 2017 via email

@timholy
Copy link
Member

timholy commented Feb 21, 2017

I think it is, but it's worth investigating further. I'm in the middle of fixing the depwarn and I'm using that version (I shouldn't do that), so right now it's ahead of master.

@Cody-G
Copy link
Contributor Author

Cody-G commented Feb 21, 2017 via email

@timholy
Copy link
Member

timholy commented Feb 21, 2017

Travis is running 0.5.0.

@Cody-G
Copy link
Contributor Author

Cody-G commented Feb 22, 2017

Very strange, I just got the same error when using the packages installed in /usr/local/julia/julia-0.5/usr/share/julia/site/v0.5/. The test that fails is register_mismatch.jl. What could we be doing differently?

Also in order to use those packages I had to run julia as super user, or else I get an error when trying to use precompiled stuff:

ERROR: LoadError: LoadError: SystemError: opening file /usr/local/julia/julia-0.5/usr/share/julia/site/lib/v0.5/Reexport.ji: Permission denied
 in #systemerror#51 at ./error.jl:34 [inlined]
 in systemerror(::String, ::Bool) at ./error.jl:34
 in open(::String, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./iostream.jl:89
 in open(::String, ::String) at ./iostream.jl:101
 in stale_cachefile(::String, ::String) at ./loading.jl:659
 in _require_search_from_serialized(::Int64, ::Symbol, ::String, ::Bool) at ./loading.jl:214
 in require(::Symbol) at ./loading.jl:371
 in include_from_node1(::String) at ./loading.jl:488
 in eval(::Module, ::Any) at ./boot.jl:234
 in require(::Symbol) at ./loading.jl:415
 in include_from_node1(::String) at ./loading.jl:488
 in process_options(::Base.JLOptions) at ./client.jl:262
 in _start() at ./client.jl:318
while loading /usr/local/julia/julia-0.5/usr/share/julia/site/v0.5/BlockRegistration/src/BlockRegistration.jl, in expression starting on line 5
while loading /usr/local/julia/julia-0.5/usr/share/julia/site/v0.5/BlockRegistration/test/runtests.jl, in expression starting on line 1

@Cody-G
Copy link
Contributor Author

Cody-G commented Feb 22, 2017

Here's my Pkg.status() when I use the shared packages on cannon. I guess we should confirm that yours looks the same...

24 required packages:
 - AxisArrays                    0.0.3
 - CUDArt                        0.2.3+             standalone
 - ColorTypes                    0.3.4
 - CoordinateTransformations     0.4.0
 - DSP                           0.1.1
 - Documenter                    0.9.0
 - GLPlot                        0.0.4
 - IJulia                        1.4.1
 - ImageMagick                   0.2.1              963ac699 (dirty)
 - Images                        0.7.0
 - Interact                      0.4.3
 - IntervalSets                  0.0.3
 - LightGraphs                   0.7.2
 - MultivariateStats             0.3.1
 - NRRD                          0.2.1
 - OffsetArrays                  0.2.14
 - PlotlyJS                      0.5.2
 - Plots                         0.10.3
 - ProfileView                   0.1.5
 - PyPlot                        2.3.1
 - Rotations                     0.3.4
 - TestImages                    0.1.3
 - Unitful                       0.0.4
 - Winston                       0.12.1
145 additional packages:
 - AffineTransforms              0.2.1
 - AutoHashEquals                0.1.1
 - AxisAlgorithms                0.1.6
 - BaseTestNext                  0.2.2
 - BinDeps                       0.4.7
 - Blink                         0.5.1
 - BlockRegistration             0.0.0-             master (unregistered)
 - BlockRegistrationScheduler    0.0.0-             master (unregistered)
 - Blosc                         0.2.0
 - BoxTrees                      0.0.0-             master (unregistered)
 - BufferedStreams               0.3.0
 - CUFFT                         0.0.4
 - Cairo                         0.2.35
 - CalciumImaging                0.0.0-             master (unregistered)
 - Calculus                      0.2.0
 - CatIndices                    0.0.2
 - CellSegmentation              0.0.0-             master (unregistered)
 - Codecs                        0.2.0
 - ColorVectorSpace              0.4.0
 - Colors                        0.7.3
 - Combinatorics                 0.3.2
 - Compat                        0.19.0
 - Compose                       0.4.5
 - ComputationalResources        0.0.2
 - Conda                         0.5.1
 - Contour                       0.2.0
 - CustomUnitRanges              0.0.4
 - DFoFArrays                    0.0.0-             master (unregistered)
 - DataArrays                    0.3.12
 - DataFrames                    0.8.5
 - DataStructures                0.5.2
 - Dates                         0.4.4
 - DiffBase                      0.0.4
 - Distances                     0.4.0
 - Distributions                 0.12.1
 - DocStringExtensions           0.3.1
 - DualNumbers                   0.2.3
 - FFTViews                      0.0.1
 - FileIO                        0.3.0
 - FixedPointNumbers             0.3.4
 - FixedSizeArrays               0.2.5
 - Formatting                    0.2.0
 - ForwardDiff                   0.3.4
 - GLAbstraction                 0.3.1
 - GLFW                          1.2.2
 - GLText                        0.0.4
 - GLWindow                      0.3.2
 - GZip                          0.2.20
 - Gadfly                        0.5.3
 - GeometryTypes                 0.2.2
 - Graphics                      0.1.3
 - Graphs                        0.7.1
 - Gtk                           0.10.4
 - GtkUtilities                  0.2.0
 - HDF5                          0.7.3
 - Hexagons                      0.0.4
 - Hiccup                        0.1.1
 - HttpCommon                    0.2.6
 - HttpParser                    0.2.0
 - HttpServer                    0.1.7
 - ImageAxes                     0.1.0
 - ImageCore                     0.1.4
 - ImageFiltering                0.1.2
 - ImageMetadata                 0.2.0
 - ImagePlayer                   0.0.0-             master (unregistered)
 - Immerse                       0.0.12
 - ImmutableArrays               0.0.12
 - IndirectArrays                0.1.1
 - IniFile                       0.2.5
 - Interpolations                0.3.7
 - Ipopt                         0.2.6
 - IterativeSolvers              0.2.2
 - Iterators                     0.3.0
 - JLD                           0.6.9
 - JSON                          0.8.3
 - JuMP                          0.15.1
 - Juno                          0.2.6
 - KernelDensity                 0.3.0
 - LaTeXStrings                  0.2.0
 - LabShare                      0.0.0-             master (unregistered)
 - Lazy                          0.11.5
 - LegacyStrings                 0.2.0
 - Libz                          0.2.4
 - LightXML                      0.4.0
 - LineSearches                  0.1.5
 - Loess                         0.1.0
 - MAT                           0.3.1
 - MacroTools                    0.3.4
 - MappedArrays                  0.0.6
 - MathProgBase                  0.5.10
 - MbedTLS                       0.4.3
 - Measures                      0.0.3
 - Media                         0.2.5
 - ModernGL                      0.1.1
 - Munkres                       0.1.0
 - Mustache                      0.1.3
 - Mux                           0.2.3
 - NLsolve                       0.9.1
 - NaNMath                       0.2.2
 - NeighborhoodClustering        0.0.0-             master (unregistered)
 - Nettle                        0.3.0
 - Optim                         0.3.1+             teh/constrained
 - PDMats                        0.5.6
 - ParserCombinator              1.7.11
 - PlotThemes                    0.1.1
 - PlotUtils                     0.3.0
 - Polynomials                   0.1.2
 - PositiveFactorizations        0.0.4
 - ProgressMeter                 0.3.3
 - PyCall                        1.10.0
 - QuadGK                        0.1.1
 - Quaternions                   0.1.1
 - RFFT                          0.0.0-             master (unregistered)
 - RangeArrays                   0.1.2
 - Ranges                        0.0.1
 - Ratios                        0.0.4
 - Reactive                      0.3.7
 - RecipesBase                   0.1.0
 - Reexport                      0.0.3
 - Requires                      0.3.0
 - ReverseDiffSparse             0.6.0
 - Rmath                         0.1.6
 - SHA                           0.3.1
 - SIUnits                       0.1.0
 - ShowItLikeYouBuildIt          0.0.1
 - Showoff                       0.0.7
 - SimpleTraits                  0.3.0
 - SortingAlgorithms             0.1.0
 - SpecialFunctions              0.1.1
 - StaticArrays                  0.2.1
 - StatsBase                     0.13.1
 - StatsFuns                     0.4.0
 - TexExtensions                 0.0.3
 - TileTrees                     0.0.0-             master (unregistered)
 - TileTreesGUI                  0.0.0-             master (unregistered)
 - TileTreesIO                   0.0.0-             master (unregistered)
 - TiledIteration                0.0.2
 - Tk                            0.4.0
 - URIParser                     0.1.8
 - VolumeRaycasting              0.0.0-             master (unregistered)
 - WebSockets                    0.2.1
 - WoodburyMatrices              0.2.2
 - ZMQ                           0.4.1
 - ZipFile                       0.3.0
 - lib                           0.0.0-             non-repo (unregistered)

@Cody-G
Copy link
Contributor Author

Cody-G commented Feb 27, 2017

So this is really bizarre. I just noticed that I get the above error when running Pkg.test but not when I directly run the tests by include("runtests.jl"). Any ideas why that could be happening? Thanks!

@timholy
Copy link
Member

timholy commented Feb 27, 2017

Ah, that's interesting. I think Pkg.test by default uses the equivalent of julia --check-bounds=yes (see the command line options with julia -h). When it's set to yes, @inbounds has no effect---array bounds are always checked. Failing to check array bounds can be dangerous: you can get a segfault if you try to access memory that's not actually allocated for the array. But in this case it's throwing an error for getindex on a UnitRange, which doesn't even have allocated memory: the core operation (omitting the bounds check) is

function getindex{T}(v::UnitRange{T}, i::Integer)
    convert(T, first(v) + i - 1)
end

which is just a simple computation, not a memory-access.

Looking at the line corresponding to the error, it's in a part of the code that is deliberately violating bounds. In block registration, of our blocks are of the same size, and each block is centered on a grid position; the grid extends all the way to the corners of the image (this makes fine-scale interpolation for warping much cleaner, we don't have to extrapolate u beyond the region where it was defined). That means the blocks at the edges of images are missing data (in 2d, half the data along an edge and 3/4 of the data at a corner). We fill those slots with NaN. But the way the code is written, it sometimes temporarily ignores their absence for the purposes of index-computation and then uses a get!(block, view(img, inds...), NaN) mechanism to extract values. It's probably throwing that error on the computation of inds.

In other words, the error you're getting does not appear to be anything to worry about: it's how the code is supposed to work. Perhaps we should rewrite that code so you can use Pkg.test (it would be nice for catching non-intentional bounds errors), but if you don't want to tackle that, for now I think you can ignore it.

@Cody-G
Copy link
Contributor Author

Cody-G commented Feb 27, 2017

I see, that all makes sense, thanks! So I guess it's conventional that Travis calls the runtests.jl script directly rather than calling Pkg.test?

I don't think I have time at the moment to update the code for Pkg.test compatibility but may return to it at some point. Would you prefer that I leave this open as a reminder? If not feel free to close. Thanks!

@timholy
Copy link
Member

timholy commented Feb 28, 2017

So I guess it's conventional that Travis calls the runtests.jl script directly rather than calling Pkg.test?

No, it usually runs Pkg.test. Hmm, I've been assuming the test failures are from running 0.5.0 rather than release-0.5, and that could still be true, but really I should modify .travis.yml so it bypasses Pkg.test.

Yes, let's leave this open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants