GPU version requirements for the search up to 10^20

Message boards : Number crunching : GPU version requirements for the search up to 10^20

To post messages, you must log in.

1 · 2 · Next

AuthorMessage
Sergei Chernykh
Project administrator
Project developer

Send message
Joined: 5 Jan 17
Posts: 518
Credit: 72,451,573
RAC: 0
   
Message 535 - Posted: 1 Jul 2017, 10:53:54 UTC
Last modified: 1 Jul 2017, 11:00:19 UTC

Current GPU version's requirements: any AMD (HD 5xxx or newer) or NVIDIA (GTX 4xx or newer) GPU with OpenCL support and at least 2 GB of video memory

If you run an AMD card on Windows, make sure you use driver version 14.12 or newer, otherwise it may fail at startup because of OpenCL code compilation error.
ID: 535 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Beyond
Avatar

Send message
Joined: 12 Apr 17
Posts: 13
Credit: 2,369,891,829
RAC: 0
   
Message 538 - Posted: 1 Jul 2017, 23:05:06 UTC - in response to Message 535.  

Current GPU version's requirements: any AMD (HD 5xxx or newer) or NVIDIA (GTX 4xx or newer) GPU with OpenCL support and at least 2 GB of video memory

If you run an AMD card on Windows, make sure you use driver version 14.12 or newer, otherwise it may fail at startup because of OpenCL code compilation error.

As of today my AMD HD7950 (3GB vram) errors the WUs. Ran fine before. Checked the driver version, it was 16.9.2. Installed the newest 17.6.2. Still just errors after about 10 seconds. No problem with any other projects. Any ideas?
ID: 538 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Beyond
Avatar

Send message
Joined: 12 Apr 17
Posts: 13
Credit: 2,369,891,829
RAC: 0
   
Message 539 - Posted: 2 Jul 2017, 0:06:40 UTC - in response to Message 538.  
Last modified: 2 Jul 2017, 0:08:02 UTC

Looks like an OpenCL problem. SDERR output:

<core_client_version>7.6.33</core_client_version>
<![CDATA[
<message>
(unknown error) - exit code -1 (0xffffffff)
</message>
<stderr_txt>
Initializing prime tables...done
OpenCL.cpp, line 294: Preferences:
<project_preferences>


<allow_non_selected_apps>1</allow_non_selected_apps>
<max_jobs>0</max_jobs>
<max_cpus>2</max_cpus>
<kernel_size_amd>20</kernel_size_amd>
<kernel_size_nvidia>21</kernel_size_nvidia>
</project_preferences>


OpenCL.cpp, line 300: Kernel size for AMD GPU has been set to 20
OpenCL.cpp, line 388: clBuildProgram returned error -11
OpenCL.cpp, line 389: Build options: -cl-fast-relaxed-math -D PQ_STRIDE_SIZE=512 -D WORK_GROUP_SIZE=128 -D PHASE2_MAX_COUNT=2097152 -D LPP=1 -D NUM_DATA_CHUNKS=1 -D CHUNK_SIZE_SHIFT=31 -D SUM_ESTIMATES_128_SHIFT=64 -Werror
OpenCL.cpp, line 390: Build log:
Warnings being treated as errors
"C:\Users\Admin\AppData\Local\Temp\OCL2800T1.cl", line 238: error: goto
statement may cause irreducible control flow
goto small_numbers;
^

1 error detected in the compilation of "C:\Users\Admin\AppData\Local\Temp\OCL2800T1.cl".
Frontend phase failed compilation.

17:54:15 (2800): called boinc_finish(-1)

</stderr_txt>
ID: 539 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Beyond
Avatar

Send message
Joined: 12 Apr 17
Posts: 13
Credit: 2,369,891,829
RAC: 0
   
Message 540 - Posted: 2 Jul 2017, 3:48:22 UTC
Last modified: 2 Jul 2017, 3:49:50 UTC

Looking through the list of top hosts, many (but not all) AMD GPUs are having the same issue. They ran v1.21 fine but error with v2.01.
ID: 540 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sergei Chernykh
Project administrator
Project developer

Send message
Joined: 5 Jan 17
Posts: 518
Credit: 72,451,573
RAC: 0
   
Message 541 - Posted: 2 Jul 2017, 6:47:51 UTC
Last modified: 2 Jul 2017, 6:52:34 UTC

Not all hosts AMD Radeon have this issue. I can find only 14 hosts out of more than 140 which were active since yesterday.

Regarding the error: I only found this, so it's definitely driver issue.

P.S. Yes, I can rewrite the code and remove that "goto", but it will slow down significantly for everyone else.

P.P.S. I actually have an idea how to solve it, I'll try it now.
ID: 541 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sergei Chernykh
Project administrator
Project developer

Send message
Joined: 5 Jan 17
Posts: 518
Credit: 72,451,573
RAC: 0
   
Message 542 - Posted: 2 Jul 2017, 8:01:19 UTC

Fixed it, version 2.02 should work fine.
ID: 542 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Beyond
Avatar

Send message
Joined: 12 Apr 17
Posts: 13
Credit: 2,369,891,829
RAC: 0
   
Message 550 - Posted: 2 Jul 2017, 22:55:10 UTC - in response to Message 542.  

Fixed it, version 2.02 should work fine.

It's fixed for me. Thanks Sergei!
ID: 550 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mal.t

Send message
Joined: 24 Feb 17
Posts: 1
Credit: 32,312,139
RAC: 0
   
Message 554 - Posted: 3 Jul 2017, 15:52:39 UTC

i used to be able to crunch on my GTX560ti but am no longer getting work units since the switch to 10^20. anyway to fix this?
ID: 554 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sergei Chernykh
Project administrator
Project developer

Send message
Joined: 5 Jan 17
Posts: 518
Credit: 72,451,573
RAC: 0
   
Message 555 - Posted: 3 Jul 2017, 15:57:27 UTC - in response to Message 554.  

i used to be able to crunch on my GTX560ti but am no longer getting work units since the switch to 10^20. anyway to fix this?

No, it can't be fixed with a major rewrite of the GPU version. It just doesn't fit in 1024 MB.
ID: 555 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
gbagar

Send message
Joined: 6 Feb 17
Posts: 5
Credit: 99,608,680
RAC: 0
   
Message 586 - Posted: 20 Jul 2017, 16:56:13 UTC

I have a GTX750 1GB vRAM mixed with a GT720 2GB vRAM and a GT730 2GB vRAM on the same computer.

The GTX750 is plugged into the tertiary slot and my 64-bit Debian Linux OS with NVIDIA Driver 375.66 rightfully detects it as GPU2.

BOINC ver 7.6, however, wrongfully detects it as GPU0 and reports ONLY this card to the Amicable Numbers Server. The Monitor is NOT connected to this card.

Consequently, the AN Server reports to me that I don't have enough vRAM to run 10^20 and refuses to send me any tasks.

I have my cc_config.xml correctly configured to exclude GPU2 from Amicable Numbers and run other projects, which it does just fine. I also have GPUs 0 and 1 correctly configured to run AN 10^20 exclusively, which they do just fine with the GTX 750 1GB vRAM physically removed from the system.

Any way to convince BOINC that my GTX750 1GB vRAM is NOT GPU0 or get the AN Server to see past it to the other cards so that I can run AN 10^20 and other projects simultaneously on the same computer?

Thanks in advance,
gbagar

Note 1: Since BOINC detected the GTX750 as GPU0 even though it is in GPU2's slot, I tried changing my cc_config.xml to exclude GPU0 from AN. That did not work.

Note 2: If I put a GT720 with only 1GB of vRAM or a GT730 with only 1GB of vRAM physically in the GPU0 slot, connect the Monitor to it, and leave the cc_config.xml file set to exclude it from AN, then BOINC rightly detects it as GPU0, reports it to the AN Server as GPU0, and the AN Server looks past it to the other cards and sends AN 10^20 tasks to them.

The trick I'm looking for is how to make this happen with the GTX750 1GB vRAM, which BTW, I really want physically in the GPU0 slot, with the Monitor connected to it, and detected as such, so that me, my OS, my Driver, BOINC, and the AN Server, all see it the same way. Besides, the 750 really runs my video better.

But, I'll take any Solution or WorkAround that gets me running AN with the 750 installed anywhere. except another computer.

More thanks in advance.
ID: 586 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sergei Chernykh
Project administrator
Project developer

Send message
Joined: 5 Jan 17
Posts: 518
Credit: 72,451,573
RAC: 0
   
Message 587 - Posted: 20 Jul 2017, 19:36:34 UTC - in response to Message 586.  
Last modified: 20 Jul 2017, 19:37:44 UTC

BOINC ver 7.6, however, wrongfully detects it as GPU0 and reports ONLY this card to the Amicable Numbers Server. The Monitor is NOT connected to this card.

I think the problem is that it doesn't detect other cards. GPU detection is 100% on client side, so you should try to wipe BOINC configs and then check client logs (Tools -> Event log in BOINC Manager) to see why it ignores some GPUs.
ID: 587 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
gbagar

Send message
Joined: 6 Feb 17
Posts: 5
Credit: 99,608,680
RAC: 0
   
Message 588 - Posted: 20 Jul 2017, 21:16:04 UTC - in response to Message 587.  

Thanks Sergei...

but the event log indicates clearly that BOINC is not ignoring ANY of my GPUs and is detecting ALL 3 cards. Additionally, the other 2 cards will run other projects with the 750 installed, just not Amicable, because all it sees is the 750 with 1GB.

As a matter of fact, my 'computers list' in my account shows the computer in question as having 3 GTX750s with 1GB. This seems to be common for all computers in the list. They all show all the cards as the the same Make and Model as GPU0, For example, one computer shows 2 GT730s, though the 2nd card is a GT720. Another shows 2 GT720s, though the other card is a GT730. Maybe this is because of the way NVIDIA puts IDs in their chips. This only case in which the Servers correctly IDs other cards, is if the other card is of a different make. The Amicable Server correctly IDs the cards on one computer as 2 NVIDIAs and 1 AMD.

I've read that BOINC assigns the latest, greatest, fastest, most capable card to GPU0 no matter in which slot it is physically installed. This is OK, as long as Project Servers can see past it and correctly identify the cards on which their project can run...in Amicable's case, the cards with 2GB vRAM. I don't know if other Projects' Servers can do this, I just know that they run on my cards without question.

I realize that writing the code to enable this on a server may be troublesome and problematic. So, I'm not complaining about this, or the new vRAM reqs. All projects have to update, upgrade, and progress as technology does and the science demands. So, I cooperate, and try to beef up my machine(s) to meet the new specs. I just want to be able to mix cards, and projects, on the same machine, and have BOINC and Project Servers work with that, as long as my hardware meets their specs.

I may be the only one with this problem, so whatever changes you think you might make to the Server may not be worth it. Consequently, I'm exploring other ways to efficiently allocate my Resources to my Projects, of which, BTW, Amicable is my Fave!

I hope you find this input valuable.

Thanks Again,
gbagar
ID: 588 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
gbagar

Send message
Joined: 6 Feb 17
Posts: 5
Credit: 99,608,680
RAC: 0
   
Message 589 - Posted: 20 Jul 2017, 22:32:01 UTC - in response to Message 588.  

Update: and Fair:

I just looked at the other GPU Projects that I run, and they all report GPUs the same way as Amicable. All of a computer's GPUs are reported as the same Make and Model as GPU0, unless they are of a different Make. So this is not an Amicable Server issue by any means, it is a universal BOINC Server issue.

So, as long as whatever BOINC sees as GPU0 meets the SysReqs for a project, it will download tasks to that computer, even if its other cards do not. But, if GPU0 does not meet the SysReqs for a Project, it will not download tasks to that computer, even if GPUs 1 & 2 & so on DO meet the SysReqs for that Project.

For me, this means that I cannot run Amicable on any computer that has a GTX750 1G vRAM, because it is the most powerful card that I have, and will always be assigned to GPU0, regardless of the slot that it is actually in. Would that there were a Flag to put in a Config File to force a Particular Card to be assigned to GPU0. I've looked for one, but don't seem to be savvy enough to find it.

As a result of this experience, I just joined the BOINC Forum. Maybe I get some help from the devs or users over there.

Thanks to Sergei, and all others who read and considered helping me with this.
gbagar
ID: 589 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sergei Chernykh
Project administrator
Project developer

Send message
Joined: 5 Jan 17
Posts: 518
Credit: 72,451,573
RAC: 0
   
Message 590 - Posted: 21 Jul 2017, 7:57:21 UTC - in response to Message 589.  

I've checked BOINC client source code, and found this in "client/gpu_nvidia.cpp":
// return 1/-1/0 if device 1 is more/less/same capable than device 2.
// factors (decreasing priority):
// - compute capability
// - software version
// - available memory
// - speed
//
// If "loose", ignore FLOPS and tolerate small memory diff
//
int nvidia_compare(COPROC_NVIDIA& c1, COPROC_NVIDIA& c2, bool loose) {
    if (c1.prop.major > c2.prop.major) return 1;
    if (c1.prop.major < c2.prop.major) return -1;
    if (c1.prop.minor > c2.prop.minor) return 1;
    if (c1.prop.minor < c2.prop.minor) return -1;
    if (c1.cuda_version > c2.cuda_version) return 1;
    if (c1.cuda_version < c2.cuda_version) return -1;
    if (loose) {
        if (c1.available_ram> 1.4*c2.available_ram) return 1;
        if (c1.available_ram < .7* c2.available_ram) return -1;
        return 0;
    }
    if (c1.available_ram > c2.available_ram) return 1;
    if (c1.available_ram < c2.available_ram) return -1;
    double s1 = c1.peak_flops;
    double s2 = c2.peak_flops;
    if (s1 > s2) return 1;
    if (s1 < s2) return -1;
    return 0;
}


You can try to move this into the beginning of "nvidia_compare":
    if (c1.available_ram > c2.available_ram) return 1;
    if (c1.available_ram < c2.available_ram) return -1;

and then build BOINC client from source.
ID: 590 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
gbagar

Send message
Joined: 6 Feb 17
Posts: 5
Credit: 99,608,680
RAC: 0
   
Message 591 - Posted: 21 Jul 2017, 13:20:56 UTC - in response to Message 590.  
Last modified: 21 Jul 2017, 13:23:53 UTC

Thanks Sergei!

Your attention to my problem both honors and delights me. I am definitely going to give this solution a try, as it is time for me, in my growth as a Linuxer, to learn to work with Source Code. Now I have an excellent reason and motivation to go ahead and do just that. Along side this...

I think I may have found a possible, simpler but less elegant, work around. I can run multiple instances of the BOINC Client on the same Host using the flag:

<allow_multiple_clients>0|1</allow_multiple_clients>

and tell each instance to ignore certain Graphics Cards using the flag:

<ignore_nvidia_dev>N</ignore_nvidia_dev>.

So, one instance will ignore GPU0 and run Amicable on the other 2 cards, and the other instance will ignore GPUs 1&2 and run other projects on GPU0.

Wish me luck.
gbagar
ID: 591 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
gbagar

Send message
Joined: 6 Feb 17
Posts: 5
Credit: 99,608,680
RAC: 0
   
Message 594 - Posted: 29 Jul 2017, 4:19:25 UTC - in response to Message 591.  

Sergei,

Thanks again for your Source Code modification suggestion. But, I regret to inform you that it did not work. BOINC continued to report my GTX 750 1G as GPU0 regardless of it's actual slot, and did not report any other installed GPUs to the Project Servers at all. So, because the Amicable Server saw less than 1840M of vRAM, it continued to deny me tasks, until...

I successfully got 2 instances of the BOINC Client running on the same computer, side by side, out of different Data Directories, with different cc_config files, each ignoring the appropriate GPU, allowing the appropriate Project Server to see only the GPU that meets the SysReqs for its tasks, so that it no longer resists downloading tasks to that GPU. I am now running Amicable Tasks on 'subordinate' GPUs. Yea! Me! And thanks to all those who put instructions online that enabled my success.

NOTE: This must be done (in Debian and derivatives anyway) with a BOINC that has been compiled on your machine from Source Code. Debian (.deb) Packages create Symbolic Links between the cc_configs in different Data Directories so that a change in one is made manifest in the other. This prevents having each instance 'ignore' the appropriate GPU. Compiling from Source prevents this Link.

So, I got a little Coding experience, and a little Compiling experience. I'm edified. Don't worry, your job is safe.

Gary
ID: 594 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Tern

Send message
Joined: 17 Feb 17
Posts: 27
Credit: 69,545,002
RAC: 9,911
   
Message 616 - Posted: 30 Aug 2017, 23:36:16 UTC

I know this is a low priority - I just got my (modified) MacPro up on Amic, and CPU tasks are running now. It appears there is no Mac version of the GPU applications... :-/

(Yeah, I know, I may be the ONLY one on this project with a Mac with an AMD card in it...)
ID: 616 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sergei Chernykh
Project administrator
Project developer

Send message
Joined: 5 Jan 17
Posts: 518
Credit: 72,451,573
RAC: 0
   
Message 617 - Posted: 1 Sep 2017, 9:24:50 UTC - in response to Message 616.  

I'll look into making Mac GPU version next week.
ID: 617 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Sergei Chernykh
Project administrator
Project developer

Send message
Joined: 5 Jan 17
Posts: 518
Credit: 72,451,573
RAC: 0
   
Message 622 - Posted: 13 Sep 2017, 11:27:54 UTC - in response to Message 616.  

I know this is a low priority - I just got my (modified) MacPro up on Amic, and CPU tasks are running now. It appears there is no Mac version of the GPU applications... :-/

(Yeah, I know, I may be the ONLY one on this project with a Mac with an AMD card in it...)

I finally figured out how to compile OpenCL version for MacOS and added AMD and NVIDIA OpenCL application versions for MacOS 10.7 or newer: https://sech.me/boinc/Amicable/apps.php
ID: 622 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile JohnMD
Avatar

Send message
Joined: 8 Jan 18
Posts: 11
Credit: 25,123,011
RAC: 0
   
Message 792 - Posted: 18 Apr 2018, 22:31:57 UTC - in response to Message 535.  

Current GPU version's requirements: any AMD (HD 5xxx or newer) or NVIDIA (GTX 4xx or newer) GPU with OpenCL support and at least 2 GB of video memory

If you run an AMD card on Windows, make sure you use driver version 14.12 or newer, otherwise it may fail at startup because of OpenCL code compilation error.


Now that WU's allocate less than 900MB, is it possible to relax the GPU memory requirement to 1GB ?
ID: 792 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
1 · 2 · Next

Message boards : Number crunching : GPU version requirements for the search up to 10^20


©2024 Sergei Chernykh