Posts by Sergei Chernykh

101) Message boards : Number crunching : Prime Numbers (Message 1263)
Posted 26 Oct 2019 by Sergei Chernykh
Post:
I prefer not to store prime table on disc as it can get corrupt/overwritten either occasionally or maliciously.
102) Message boards : News : The search up to 10^20 (Message 1250)
Posted 25 Oct 2019 by Sergei Chernykh
Post:
I repeat: these 8 GB can mostly go into swap file without hurting performance. You don't need physical 8 GB RAM per task, OS will swap out unused parts of memory if you have not enough RAM.
103) Message boards : News : The search up to 10^21 (Message 1246)
Posted 25 Oct 2019 by Sergei Chernykh
Post:
The search up to 1021 has started! Over 50,000 new amicable pairs have been found in the first few days.

Due to the huge size of search space, first stage of the search will only look for all amicable pairs where smaller number is of the form 3N*...*p where N > 0 and p < 1011. I expect that this will find 2-3 million new amicable pairs in a year or so.

Note that system memory requirements are higher now because all prime numbers < 1011 need to be stored in memory for the search.
104) Message boards : News : The search up to 10^20 (Message 1245)
Posted 25 Oct 2019 by Sergei Chernykh
Post:
It doesn't require 8 GB all the time, you could increase your swap file and be fine with ~1.5 GB physical RAM per task. Now I'm not sure how to make BOINC run with more memory used than available.
105) Message boards : News : The search up to 10^20 (Message 1235)
Posted 23 Oct 2019 by Sergei Chernykh
Post:
Running GPU on 1021 is fine now. Even 1 task per GPU will get you almost 100% GPU load (except for when task starts), 2 tasks per GPU will guarantee you 100% load all the time. CPU usage of GPU tasks is low, so you can run CPU tasks on the same machine.
106) Message boards : News : The search up to 10^20 (Message 1233)
Posted 21 Oct 2019 by Sergei Chernykh
Post:
When can we expect more of the 10^21 tasks?

As soon as 10^20 tasks run out or a bit before that for a smooth transition.
107) Message boards : News : The search up to 10^20 (Message 1230)
Posted 21 Oct 2019 by Sergei Chernykh
Post:
I pushed search limits for 1021 a bit further into large primes territory, so memory requirements are higher now (all primes up to 1011 are checked). CPU version requires 2.85 GB of system memory. GPU version fits in 1 GB on GPU, but requires 8 GB of system memory after refactoring.
108) Message boards : News : The search up to 10^20 (Message 1227)
Posted 19 Oct 2019 by Sergei Chernykh
Post:
There's literally only 310 such tasks for the entire search. lpp stands for "largest prime power" and when it's 2 or 3, you only get to check sqrt(N) or cuberoot(N) numbers. So we shouldn't bother with them at all. But they are also good for testing and finding bugs, so I added them for the beta test.
109) Message boards : News : The search up to 10^20 (Message 1224)
Posted 19 Oct 2019 by Sergei Chernykh
Post:
I've checked it. Tasks with "/lpp" in command line iterate over many more ranges than normal tasks and many of these ranges end up with 0 numbers for GPU. This is why it uses only CPU in the end and is at 100% - it counts only how many numbers to check on GPU.
110) Message boards : News : The search up to 10^20 (Message 1221)
Posted 19 Oct 2019 by Sergei Chernykh
Post:
Probably task_size parameter was calculated incorrectly, so it can count to 100% before the actual work is done. I'll check this task.
111) Message boards : News : The search up to 10^20 (Message 1219)
Posted 19 Oct 2019 by Sergei Chernykh
Post:
And there is a bug in OpenCL version: https://sech.me/boinc/Amicable/workunit.php?wuid=11571043
I reproduced it locally. Kernel size = 23 skips some numbers, kernel size = 21 works fine. But other work units might behave differently, i.e. kernel size 21 will skip numbers.

P.S. Fixed in version 3.01. It was a bug introduced during refactoring, so 1020 search wasn't affected.
112) Message boards : News : The search up to 10^20 (Message 1218)
Posted 18 Oct 2019 by Sergei Chernykh
Post:
The work units with "/lpp" in command line are special ones, they're more hard on CPU, so 100% GPU load is not guaranteed. But there is only a few of them.

And yes, new pairs: https://sech.me/ap/log/2019/2019-10-19/VOLUNTEERS%40BOINC.txt
113) Message boards : News : The search up to 10^20 (Message 1216)
Posted 18 Oct 2019 by Sergei Chernykh
Post:
If there is some task that doesn't finish on GPU, give me a link to it and I'll check it. And I see that there are already some new pairs found.
114) Message boards : News : The search up to 10^20 (Message 1211)
Posted 18 Oct 2019 by Sergei Chernykh
Post:
I've added applications for 1021, set it as a beta for now and added 1000 first work units for testing. We'll see how it goes.

P.S. But all amicable pairs found during this test will be counted as usual.
115) Message boards : News : The search up to 10^20 (Message 1209)
Posted 17 Oct 2019 by Sergei Chernykh
Post:
Right now it takes 5-10 seconds to generate a task and it's a sequential algorithm - every next range must start where the previous range ended, and I have no way to determine range borders until I run a quick emulation of the search. I can split it into several parallel processes however - for example, N processes for ranges starting with 3N, but each process is also taking 8.5 GB of memory for storing prime numbers. I'll see if I can share these 8.5 GB between processes and run task generators on all 16 threads.

P.S. I think that even now I'm able to generate tasks faster than the crunchers will be able to consume them, so it's not really a problem.
116) Message boards : News : The search up to 10^20 (Message 1207)
Posted 16 Oct 2019 by Sergei Chernykh
Post:
Yes, it was intentional. I need a bit more time to generate work units for 1021. Everything is ready, but task generation is sloooow.... I hope I'll be able to provide around 10,000 WUs next week when the new search starts.
117) Message boards : News : The search up to 10^20 (Message 1204)
Posted 14 Oct 2019 by Sergei Chernykh
Post:
Yes, I'll prepare CPU and GPU applications for 10^21 this week. GPU version will need a bit of refactoring because as of now it requires more than 8 GB of GPU memory for 10^21. I'll have to split workunits into 8 stages (1 GB each) for GPU.
118) Message boards : Number crunching : A few questions regarding my hardware. (Message 1189)
Posted 29 Aug 2019 by Sergei Chernykh
Post:
I plan to add 1021 app before October, I just haven't figured out the optimal way to run 1021 search yet. I need to go through current search results and find the most juicy subsets of tasks to run first. I think something like "searching through all odd numbers with largest prime factor < 1011" can yield over a million new pairs in a year or two.
119) Message boards : Number crunching : Question About The Ranges Tested Multiple Times (Message 1174)
Posted 19 Jun 2019 by Sergei Chernykh
Post:
Yes, it should move faster now, maybe 1020 will even finish before the end of this summer.
120) Message boards : Number crunching : Question About The Ranges Tested Multiple Times (Message 1171)
Posted 19 Jun 2019 by Sergei Chernykh
Post:
Yes, it sends smaller batches to CharityEngine's computers because they're mostly unstable/unreliable. But it shouldn't send repeating tasks, I'll look into it.

Edit: yes, there was a bug in the code. I forgot to move the batch from "unsent" to "sent" folder in this case. Thanks for noticing! But anyway, it didn't go completely to waste because it provided some more redundancy in the search up to this point.
121) Message boards : Number crunching : Maximum Prime Size for Large Prime Workunits (Message 1146)
Posted 24 May 2019 by Sergei Chernykh
Post:
I've just checked "unsent WUs" directory on the server: there are 705,478 WU left right now.
122) Message boards : Getting started : Unable to add admin (Message 1143)
Posted 23 May 2019 by Sergei Chernykh
Post:
Where exactly do you get this error?
123) Message boards : Number crunching : Maximum Prime Size for Large Prime Workunits (Message 1137)
Posted 19 May 2019 by Sergei Chernykh
Post:
The largest prime will be 5E18 (1E20 / 20) because numbers of the form N*P where N < 20 and P is prime can't be amicable.
124) Message boards : News : GPU version bugfix release (Message 1126)
Posted 8 Apr 2019 by Sergei Chernykh
Post:
GPU version is limited by CPU now, so it's not correct to compare two powerful GPUs here. You're basically comparing CPUs.
125) Message boards : News : GPU version bugfix release (Message 1122)
Posted 8 Apr 2019 by Sergei Chernykh
Post:
GPU versions for all platforms were updated to fix "Error -58" computation error when resuming stopped task.
126) Message boards : Bug tracker : Computation Error Upon Suspension and Resumption of GPU WU (Message 1121)
Posted 8 Apr 2019 by Sergei Chernykh
Post:
Yes, I fixed it a few minutes ago: https://github.com/SChernykh/Amicable/commit/806085804dc51e48aef2527cd18861ad3a986bc0

I'll update GPU versions today.
127) Message boards : Number crunching : Computation error (Message 1119)
Posted 8 Apr 2019 by Sergei Chernykh
Post:
It looks like I fixed this error: https://github.com/SChernykh/Amicable/commit/806085804dc51e48aef2527cd18861ad3a986bc0

I'll test it some more and update GPU versions today.
128) Message boards : Number crunching : Computation error (Message 1110)
Posted 1 Apr 2019 by Sergei Chernykh
Post:
I don't really know what causes error -58 (CL_INVALID_EVENT). It's triggered at this line: https://github.com/SChernykh/Amicable/blob/boinc-opencl-version-128-bit/Amicable/OpenCL.cpp#L1017 - but it's always set properly in the preceding call to clEnqueueNDRangeKernel on the last iteration of "for" loop. My guess is that OpenCL driver runs out of resources occasionally.
129) Message boards : Number crunching : Computation error (Message 1099)
Posted 26 Feb 2019 by Sergei Chernykh
Post:
clEnqueueWriteBuffer returned error -5

This is CL_OUT_OF_RESOURCES error. Try to reduce kernel size in computing preferences: https://sech.me/boinc/Amicable/prefs.php?subset=project
130) Message boards : Number crunching : Computers list is gone? (Message 1096)
Posted 22 Feb 2019 by Sergei Chernykh
Post:
You didn't lose credit, it's saved separately in your user account.
131) Message boards : Number crunching : GPU Utilization (Message 1086)
Posted 17 Feb 2019 by Sergei Chernykh
Post:
Reduce kernel size, run as many WUs per GPU as possible. 8 WUs per GPU should be good enough to load it.
132) Message boards : Number crunching : Computing Preferences (Message 1068)
Posted 21 Jan 2019 by Sergei Chernykh
Post:
I changed nothing server-side recently. Option to not use CPUs should be available on preferences page: https://sech.me/boinc/Amicable/prefs.php?subset=project
133) Message boards : Random stuff : Project Status Page (Message 1066)
Posted 21 Jan 2019 by Sergei Chernykh
Post:
Both CPUs are still in that list.
15 Reference CPU: Intel Core i7-5960X (8 cores, 16 threads, 3.0 - 3.5 GHz) 1.236 2,537.0
73 Reference CPU: Intel Core i7-4770K (4 cores, 8 threads, 3.9 GHz) 0.577 5,435.7
134) Message boards : Number crunching : Why are the RTX Graphics Cards now really Poor at running this Project. (Message 1063)
Posted 19 Jan 2019 by Sergei Chernykh
Post:
Why are the tasks not just ran on the CPUs?

Because they are GPU tasks? The actual search is still done on GPU, it just needs a list of prime numbers in given interval to proceed, and this is done on CPU.
135) Message boards : Number crunching : Why are the RTX Graphics Cards now really Poor at running this Project. (Message 1060)
Posted 18 Jan 2019 by Sergei Chernykh
Post:
The bottleneck is prime sieving at this stage because it's done on CPU. The only solution is to run as many tasks per GPU as possible - reduce kernel size to fit more tasks.
136) Message boards : Bug tracker : GTX 465? (Message 1049)
Posted 2 Jan 2019 by Sergei Chernykh
Post:
CL_DEVICE_MAX_MEM_ALLOC_SIZE is less than 256 MB

Your GPU drivers can't allocate 256 MB of memory for OpenCL applications in one go, so Amicable Numbers can't run.
137) Message boards : Number crunching : GPU version requirements for the search up to 10^20 (Message 1048)
Posted 2 Jan 2019 by Sergei Chernykh
Post:
It was done. Current requirement is 800 MB GPU memory to run a task.
138) Message boards : Number crunching : Amicable Numbers has slowed down (Message 1014)
Posted 9 Dec 2018 by Sergei Chernykh
Post:
I checked "good" and "bad" WUs in profiler, and got this for "good" WU:
InitialCheck128() = 22.36% of execution time
CheckPairInternal() = 72.05% of execution time

and this for "bad" WU:
InitialCheck128() = 83.53% of execution time
CheckPairInternal() = 15.37% of execution time

It looks like some number ranges require much more time to switch from slow 128-bit trial divisions to fast 64-bit code. There is nothing wrong with the code itself, it's just my assumption that all "large prime" ranges are the same turned out to be wrong. What's interesting is that OpenCL version doesn't suffer from this problem - probably because they're 32-bit natively and difference between 64 and 128-bit code is much smaller there.
139) Message boards : News : The search up to 10^20 (Message 1008)
Posted 2 Dec 2018 by Sergei Chernykh
Post:
I think it makes sense now to start the search up to 1021 once current search completes. We can start it from "rich" ranges of numbers and find 2-3 millions new amicable pairs within first year, according to my estimations.
140) Message boards : Number crunching : Countermeasures for Increased CPU Time (Message 1003)
Posted 27 Nov 2018 by Sergei Chernykh
Post:
i see increasing gpu memory use to 3900-4000 mb And gpu load fall litle bit to 85-90% always on kernel 23

Don't use kernel size 23 with multiple tasks, kernel size=21 will do just fine. Memory usage shouldn't be increasing at this stage of the search, it mainly depends on kernel size you select.
141) Message boards : Number crunching : Amicable Numbers has slowed down (Message 973)
Posted 2 Nov 2018 by Sergei Chernykh
Post:
All CPU tasks should be balanced to roughly the same execution time, they were generated with primesieve performance taken into account. it's only OpenCL version that will slowdown eventually.
142) Message boards : Bug tracker : Amicable Jobs not throttled? (Message 971)
Posted 2 Nov 2018 by Sergei Chernykh
Post:
There was a rare random bug (validation errors) that disappeared when I turned off throttling for CPU tasks. I'll double check it when I prepare new app versions with updated primesieve this month.
143) Message boards : Number crunching : Amicable Numbers has slowed down (Message 970)
Posted 2 Nov 2018 by Sergei Chernykh
Post:
Oh my... I guess I'll need to update primesieve in all versions soon. This month probably because it's also used in OpenCL versions and will become a bottleneck as numbers grow larger.
144) Message boards : Number crunching : Amicable Numbers has slowed down (Message 964)
Posted 1 Nov 2018 by Sergei Chernykh
Post:
My only guess is as numbers grow larger, internal buffers don't fit in CPU cache anymore. Hence the slowdown.
145) Message boards : Number crunching : Problem with valid wus count on All Tasks page? (Message 958)
Posted 30 Oct 2018 by Sergei Chernykh
Post:
Old WUs get deleted 1 week after they are verified.
146) Message boards : Number crunching : Probability (Message 955)
Posted 29 Oct 2018 by Sergei Chernykh
Post:
This page shows it: https://sech.me/boinc/Amicable/server_status.php
ETA at current speed (search up to 1020) Mar 2019
147) Message boards : Number crunching : Probability (Message 953)
Posted 28 Oct 2018 by Sergei Chernykh
Post:
There are 4,197,265 known amicable pairs below 1020 or approximately 1 pair per 23.825*1012 numbers. So the probability is low, very low if you just check numbers one by one. If you search them in more clever order, it can be much higher for some ranges. This project was founding thousands pairs per day at some stage, but now it's chewing through last ranges which are unlikely to have new pairs.
148) Message boards : Number crunching : Work unit errors (Message 943)
Posted 4 Oct 2018 by Sergei Chernykh
Post:
It can happen if:
- this is an old work unit which expired for someone else
- then it was sent to you
- then this "someone else" finally sends it to the server
- the server validates it and cancels all remaining "in progress" tasks (including yours)

P.S. But your 3 errors are just computing errors, they didn't happen immediately after start.
149) Message boards : Number crunching : Work unit errors (Message 941)
Posted 4 Oct 2018 by Sergei Chernykh
Post:
I see only 3 errors out of 925 work units for your PC. Time limit for work unit is 3 days from the time it was sent to client.
150) Message boards : Number crunching : GPU version: kernel size tuning and less UI lags (Message 934)
Posted 29 Sep 2018 by Sergei Chernykh
Post:
From what I heard, RTX series has separate integer and FP units, and this project is integer only. This is probably why RTX gets such boost.


Previous 50 · Next 50


©2024 Sergei Chernykh