[Rtk-users] using >1 video cards

[Simon Rit]

Hi,
Interesting! I have never worked on that so... good luck! I have no idea
why you get this error, maybe one of the filter is not threadsafe? But in
my experience, ITK filters can be used like this. Maybe there is one issue
in our CUDA framework which has never been tested in such a manner.
I think you don't have to do your grep but replace the line
    m_DeviceIdx = itk::CudaGetMaxFlopsDev();
with the device of your choice in
utilities/ITKCudaCommon/src/itkCudaContextManager.cxx. All the other
cuCtxSetCurrent depend on that choice. What I would have done is changed
this via an (optional) env variable and run rtkfdk twice to create each
subvolume, each with a different thread.
Good luck and keep us posted if you have a working solution.
Simon

On Sat, Oct 15, 2016 at 9:30 AM, Boris Lahmadulin <zagahulik at gmail.com>
wrote:

> Hello. I work with high resolution projections (32 mp) and reconstuction
> takes some longstanding time. At the same time there are two video cards in
> my computer, but uses only one. I wonder if i can use two video cards to
> reduce reconstruction time.
> What i tried to do in short:
>
> - execute this script on newest rtk revisiion
> for ($i in $(grep -nrl "cuCtxSetCurrent" utilities/ITKCudaCommon/*))
>     echo "!$i"
>     p="_"
>     grep -vE "(cuCtxSetCurrent)" $i > $i$p
>     mv $i$p $i
> done
> i.e. remove all lines where happens setting of video card context
>
> - in rtkfdk.cxx by using posix threads create two threads, where i set
> contexts through cuCtxSetCurrent(0) and cuCtxSetCurrentset(1)
> correspondingly and wait for signals;
> - in main thread for each working thread it created separated graphs, with
> reader, ddf, pssf, constant   (which is splitted by two, i.e. dimension[1]
> /= 2; origin+= ), fdk.
> - then fdk pointers put to shared data, so working threads can use them.
> After all signal is emmited  and fdk->update() is called at working threads
> - since job is done, i copy two resulted subvolumes to one and save.
>
> it works and reconstuction time is reduced for about 1.5 times! However,
> if i run it a lot of times, finally it crashes with
>
> img->GetRequestedRegion() = [ img->GetRequestedRegion() = [
> ImageRegionImageRegion ( (0x7f45f02293600x7f45f0229360)
> )
>   Dimension:   Dimension: 0x30x3
>   Index:
>   Index: [[00, , 0x14d0x14d, , 0xcf0xcf]]
>   Size:
>   Size: [[0x3e8, 0x14f, 0x3e8, 0x1]0x14f,
>  ]
> img->GetLargestPossibleRegion() = [ 0x1ImageRegion (]0x7f45f0229328)
>
>   Dimension:  ]
> img->GetLargestPossibleRegion() = [ ImageRegion (0x30x7f45f0229328
> )
>   Index: [0, 0, 0]
>   Size: [0x3e8, 0x29a, 0x1bc]
>  ]
>   Dimension: 0x3
>   Index: [0, 0, 0]
>   Size: [0x3e8, 0x29a, 0x1bc]
>  ]
> ExceptionObject caught with arg->fdk->Update() in file rtkfdk.cxx line
> ExceptionObject caught with arg->fdk->Update() in file 242
> rtkfdk.cxx line 242
>
>
> itk::InvalidRequestedRegionError (itk::InvalidRequestedRegionError
> (0x7f45d048a100)
> 0x7f45d85a9ee0)
> Location: "unknown"
> Location: "unknown"
> File: /home/koza/thirdparty/InsightToolkit-4.8.0/Modules/
> Core/Common/src/itkDataObject.cxx
> Line: File: /home/koza/thirdparty/InsightToolkit-4.8.0/Modules/
> Core/Common/src/itkDataObject.cxx
> Line: 393
> Description: Requested region is (at least partially) outside the largest
> possible region.
>
> 393
> Description: Requested region is (at least partially) outside the largest
> possible region.
>
>
> It seems that itk classes are not reenterable, Is there way to fix it ? Or
> is there another way to make several video cards using for reconstuction in
> parallel ?
>
> Thanks.
>
> _______________________________________________
> Rtk-users mailing list
> Rtk-users at public.kitware.com
> http://public.kitware.com/mailman/listinfo/rtk-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.creatis.insa-lyon.fr/pipermail/rtk-users/attachments/20161017/574dd8aa/attachment.htm>