Message boards : Number crunching : GPU version requirements for the search up to 10^20
Author | Message |
---|---|
Sergei Chernykh Project administrator Project developer Send message Joined: 5 Jan 17 Posts: 534 Credit: 72,451,573 RAC: 0 |
Current GPU version's requirements: any AMD (HD 5xxx or newer) or NVIDIA (GTX 4xx or newer) GPU with OpenCL support and at least 2 GB of video memory If you run an AMD card on Windows, make sure you use driver version 14.12 or newer, otherwise it may fail at startup because of OpenCL code compilation error. |
Beyond Send message Joined: 12 Apr 17 Posts: 13 Credit: 2,369,891,829 RAC: 0 |
Current GPU version's requirements: any AMD (HD 5xxx or newer) or NVIDIA (GTX 4xx or newer) GPU with OpenCL support and at least 2 GB of video memory As of today my AMD HD7950 (3GB vram) errors the WUs. Ran fine before. Checked the driver version, it was 16.9.2. Installed the newest 17.6.2. Still just errors after about 10 seconds. No problem with any other projects. Any ideas? |
Beyond Send message Joined: 12 Apr 17 Posts: 13 Credit: 2,369,891,829 RAC: 0 |
Looks like an OpenCL problem. SDERR output: <core_client_version>7.6.33</core_client_version> <![CDATA[ <message> (unknown error) - exit code -1 (0xffffffff) </message> <stderr_txt> Initializing prime tables...done OpenCL.cpp, line 294: Preferences: <project_preferences> <allow_non_selected_apps>1</allow_non_selected_apps> <max_jobs>0</max_jobs> <max_cpus>2</max_cpus> <kernel_size_amd>20</kernel_size_amd> <kernel_size_nvidia>21</kernel_size_nvidia> </project_preferences> OpenCL.cpp, line 300: Kernel size for AMD GPU has been set to 20 OpenCL.cpp, line 388: clBuildProgram returned error -11 OpenCL.cpp, line 389: Build options: -cl-fast-relaxed-math -D PQ_STRIDE_SIZE=512 -D WORK_GROUP_SIZE=128 -D PHASE2_MAX_COUNT=2097152 -D LPP=1 -D NUM_DATA_CHUNKS=1 -D CHUNK_SIZE_SHIFT=31 -D SUM_ESTIMATES_128_SHIFT=64 -Werror OpenCL.cpp, line 390: Build log: Warnings being treated as errors "C:\Users\Admin\AppData\Local\Temp\OCL2800T1.cl", line 238: error: goto statement may cause irreducible control flow goto small_numbers; ^ 1 error detected in the compilation of "C:\Users\Admin\AppData\Local\Temp\OCL2800T1.cl". Frontend phase failed compilation. 17:54:15 (2800): called boinc_finish(-1) </stderr_txt> |
Beyond Send message Joined: 12 Apr 17 Posts: 13 Credit: 2,369,891,829 RAC: 0 |
Looking through the list of top hosts, many (but not all) AMD GPUs are having the same issue. They ran v1.21 fine but error with v2.01. |
Sergei Chernykh Project administrator Project developer Send message Joined: 5 Jan 17 Posts: 534 Credit: 72,451,573 RAC: 0 |
Not all hosts AMD Radeon have this issue. I can find only 14 hosts out of more than 140 which were active since yesterday. Regarding the error: I only found this, so it's definitely driver issue. P.S. Yes, I can rewrite the code and remove that "goto", but it will slow down significantly for everyone else. P.P.S. I actually have an idea how to solve it, I'll try it now. |
Sergei Chernykh Project administrator Project developer Send message Joined: 5 Jan 17 Posts: 534 Credit: 72,451,573 RAC: 0 |
Fixed it, version 2.02 should work fine. |
Beyond Send message Joined: 12 Apr 17 Posts: 13 Credit: 2,369,891,829 RAC: 0 |
Fixed it, version 2.02 should work fine. It's fixed for me. Thanks Sergei! |
mal.t Send message Joined: 24 Feb 17 Posts: 1 Credit: 32,312,139 RAC: 0 |
i used to be able to crunch on my GTX560ti but am no longer getting work units since the switch to 10^20. anyway to fix this? |
Sergei Chernykh Project administrator Project developer Send message Joined: 5 Jan 17 Posts: 534 Credit: 72,451,573 RAC: 0 |
i used to be able to crunch on my GTX560ti but am no longer getting work units since the switch to 10^20. anyway to fix this? No, it can't be fixed with a major rewrite of the GPU version. It just doesn't fit in 1024 MB. |
gbagar Send message Joined: 6 Feb 17 Posts: 5 Credit: 99,608,680 RAC: 0 |
I have a GTX750 1GB vRAM mixed with a GT720 2GB vRAM and a GT730 2GB vRAM on the same computer. The GTX750 is plugged into the tertiary slot and my 64-bit Debian Linux OS with NVIDIA Driver 375.66 rightfully detects it as GPU2. BOINC ver 7.6, however, wrongfully detects it as GPU0 and reports ONLY this card to the Amicable Numbers Server. The Monitor is NOT connected to this card. Consequently, the AN Server reports to me that I don't have enough vRAM to run 10^20 and refuses to send me any tasks. I have my cc_config.xml correctly configured to exclude GPU2 from Amicable Numbers and run other projects, which it does just fine. I also have GPUs 0 and 1 correctly configured to run AN 10^20 exclusively, which they do just fine with the GTX 750 1GB vRAM physically removed from the system. Any way to convince BOINC that my GTX750 1GB vRAM is NOT GPU0 or get the AN Server to see past it to the other cards so that I can run AN 10^20 and other projects simultaneously on the same computer? Thanks in advance, gbagar Note 1: Since BOINC detected the GTX750 as GPU0 even though it is in GPU2's slot, I tried changing my cc_config.xml to exclude GPU0 from AN. That did not work. Note 2: If I put a GT720 with only 1GB of vRAM or a GT730 with only 1GB of vRAM physically in the GPU0 slot, connect the Monitor to it, and leave the cc_config.xml file set to exclude it from AN, then BOINC rightly detects it as GPU0, reports it to the AN Server as GPU0, and the AN Server looks past it to the other cards and sends AN 10^20 tasks to them. The trick I'm looking for is how to make this happen with the GTX750 1GB vRAM, which BTW, I really want physically in the GPU0 slot, with the Monitor connected to it, and detected as such, so that me, my OS, my Driver, BOINC, and the AN Server, all see it the same way. Besides, the 750 really runs my video better. But, I'll take any Solution or WorkAround that gets me running AN with the 750 installed anywhere. except another computer. More thanks in advance. |
Sergei Chernykh Project administrator Project developer Send message Joined: 5 Jan 17 Posts: 534 Credit: 72,451,573 RAC: 0 |
BOINC ver 7.6, however, wrongfully detects it as GPU0 and reports ONLY this card to the Amicable Numbers Server. The Monitor is NOT connected to this card. I think the problem is that it doesn't detect other cards. GPU detection is 100% on client side, so you should try to wipe BOINC configs and then check client logs (Tools -> Event log in BOINC Manager) to see why it ignores some GPUs. |
gbagar Send message Joined: 6 Feb 17 Posts: 5 Credit: 99,608,680 RAC: 0 |
Thanks Sergei... but the event log indicates clearly that BOINC is not ignoring ANY of my GPUs and is detecting ALL 3 cards. Additionally, the other 2 cards will run other projects with the 750 installed, just not Amicable, because all it sees is the 750 with 1GB. As a matter of fact, my 'computers list' in my account shows the computer in question as having 3 GTX750s with 1GB. This seems to be common for all computers in the list. They all show all the cards as the the same Make and Model as GPU0, For example, one computer shows 2 GT730s, though the 2nd card is a GT720. Another shows 2 GT720s, though the other card is a GT730. Maybe this is because of the way NVIDIA puts IDs in their chips. This only case in which the Servers correctly IDs other cards, is if the other card is of a different make. The Amicable Server correctly IDs the cards on one computer as 2 NVIDIAs and 1 AMD. I've read that BOINC assigns the latest, greatest, fastest, most capable card to GPU0 no matter in which slot it is physically installed. This is OK, as long as Project Servers can see past it and correctly identify the cards on which their project can run...in Amicable's case, the cards with 2GB vRAM. I don't know if other Projects' Servers can do this, I just know that they run on my cards without question. I realize that writing the code to enable this on a server may be troublesome and problematic. So, I'm not complaining about this, or the new vRAM reqs. All projects have to update, upgrade, and progress as technology does and the science demands. So, I cooperate, and try to beef up my machine(s) to meet the new specs. I just want to be able to mix cards, and projects, on the same machine, and have BOINC and Project Servers work with that, as long as my hardware meets their specs. I may be the only one with this problem, so whatever changes you think you might make to the Server may not be worth it. Consequently, I'm exploring other ways to efficiently allocate my Resources to my Projects, of which, BTW, Amicable is my Fave! I hope you find this input valuable. Thanks Again, gbagar |
gbagar Send message Joined: 6 Feb 17 Posts: 5 Credit: 99,608,680 RAC: 0 |
Update: and Fair: I just looked at the other GPU Projects that I run, and they all report GPUs the same way as Amicable. All of a computer's GPUs are reported as the same Make and Model as GPU0, unless they are of a different Make. So this is not an Amicable Server issue by any means, it is a universal BOINC Server issue. So, as long as whatever BOINC sees as GPU0 meets the SysReqs for a project, it will download tasks to that computer, even if its other cards do not. But, if GPU0 does not meet the SysReqs for a Project, it will not download tasks to that computer, even if GPUs 1 & 2 & so on DO meet the SysReqs for that Project. For me, this means that I cannot run Amicable on any computer that has a GTX750 1G vRAM, because it is the most powerful card that I have, and will always be assigned to GPU0, regardless of the slot that it is actually in. Would that there were a Flag to put in a Config File to force a Particular Card to be assigned to GPU0. I've looked for one, but don't seem to be savvy enough to find it. As a result of this experience, I just joined the BOINC Forum. Maybe I get some help from the devs or users over there. Thanks to Sergei, and all others who read and considered helping me with this. gbagar |
Sergei Chernykh Project administrator Project developer Send message Joined: 5 Jan 17 Posts: 534 Credit: 72,451,573 RAC: 0 |
I've checked BOINC client source code, and found this in "client/gpu_nvidia.cpp": // return 1/-1/0 if device 1 is more/less/same capable than device 2. // factors (decreasing priority): // - compute capability // - software version // - available memory // - speed // // If "loose", ignore FLOPS and tolerate small memory diff // int nvidia_compare(COPROC_NVIDIA& c1, COPROC_NVIDIA& c2, bool loose) { if (c1.prop.major > c2.prop.major) return 1; if (c1.prop.major < c2.prop.major) return -1; if (c1.prop.minor > c2.prop.minor) return 1; if (c1.prop.minor < c2.prop.minor) return -1; if (c1.cuda_version > c2.cuda_version) return 1; if (c1.cuda_version < c2.cuda_version) return -1; if (loose) { if (c1.available_ram> 1.4*c2.available_ram) return 1; if (c1.available_ram < .7* c2.available_ram) return -1; return 0; } if (c1.available_ram > c2.available_ram) return 1; if (c1.available_ram < c2.available_ram) return -1; double s1 = c1.peak_flops; double s2 = c2.peak_flops; if (s1 > s2) return 1; if (s1 < s2) return -1; return 0; } You can try to move this into the beginning of "nvidia_compare": if (c1.available_ram > c2.available_ram) return 1; if (c1.available_ram < c2.available_ram) return -1; and then build BOINC client from source. |
gbagar Send message Joined: 6 Feb 17 Posts: 5 Credit: 99,608,680 RAC: 0 |
Thanks Sergei! Your attention to my problem both honors and delights me. I am definitely going to give this solution a try, as it is time for me, in my growth as a Linuxer, to learn to work with Source Code. Now I have an excellent reason and motivation to go ahead and do just that. Along side this... I think I may have found a possible, simpler but less elegant, work around. I can run multiple instances of the BOINC Client on the same Host using the flag: <allow_multiple_clients>0|1</allow_multiple_clients> and tell each instance to ignore certain Graphics Cards using the flag: <ignore_nvidia_dev>N</ignore_nvidia_dev>. So, one instance will ignore GPU0 and run Amicable on the other 2 cards, and the other instance will ignore GPUs 1&2 and run other projects on GPU0. Wish me luck. gbagar |
gbagar Send message Joined: 6 Feb 17 Posts: 5 Credit: 99,608,680 RAC: 0 |
Sergei, Thanks again for your Source Code modification suggestion. But, I regret to inform you that it did not work. BOINC continued to report my GTX 750 1G as GPU0 regardless of it's actual slot, and did not report any other installed GPUs to the Project Servers at all. So, because the Amicable Server saw less than 1840M of vRAM, it continued to deny me tasks, until... I successfully got 2 instances of the BOINC Client running on the same computer, side by side, out of different Data Directories, with different cc_config files, each ignoring the appropriate GPU, allowing the appropriate Project Server to see only the GPU that meets the SysReqs for its tasks, so that it no longer resists downloading tasks to that GPU. I am now running Amicable Tasks on 'subordinate' GPUs. Yea! Me! And thanks to all those who put instructions online that enabled my success. NOTE: This must be done (in Debian and derivatives anyway) with a BOINC that has been compiled on your machine from Source Code. Debian (.deb) Packages create Symbolic Links between the cc_configs in different Data Directories so that a change in one is made manifest in the other. This prevents having each instance 'ignore' the appropriate GPU. Compiling from Source prevents this Link. So, I got a little Coding experience, and a little Compiling experience. I'm edified. Don't worry, your job is safe. Gary |
Tern Send message Joined: 17 Feb 17 Posts: 27 Credit: 69,545,002 RAC: 13 |
I know this is a low priority - I just got my (modified) MacPro up on Amic, and CPU tasks are running now. It appears there is no Mac version of the GPU applications... :-/ (Yeah, I know, I may be the ONLY one on this project with a Mac with an AMD card in it...) |
Sergei Chernykh Project administrator Project developer Send message Joined: 5 Jan 17 Posts: 534 Credit: 72,451,573 RAC: 0 |
I'll look into making Mac GPU version next week. |
Sergei Chernykh Project administrator Project developer Send message Joined: 5 Jan 17 Posts: 534 Credit: 72,451,573 RAC: 0 |
I know this is a low priority - I just got my (modified) MacPro up on Amic, and CPU tasks are running now. It appears there is no Mac version of the GPU applications... :-/ I finally figured out how to compile OpenCL version for MacOS and added AMD and NVIDIA OpenCL application versions for MacOS 10.7 or newer: https://sech.me/boinc/Amicable/apps.php |
JohnMD Send message Joined: 8 Jan 18 Posts: 11 Credit: 25,123,011 RAC: 0 |
Current GPU version's requirements: any AMD (HD 5xxx or newer) or NVIDIA (GTX 4xx or newer) GPU with OpenCL support and at least 2 GB of video memory Now that WU's allocate less than 900MB, is it possible to relax the GPU memory requirement to 1GB ? |
Message boards : Number crunching : GPU version requirements for the search up to 10^20
©2024 Sergei Chernykh