haskell - Parallel processing in conduit flow -

-

I really like the concept of a conduit / pipes to implement the operation for a streaming IO source. I'm interested in building tools that work on very large log files, an easy way to write a parallel code of attraction from Pisson / Ruby to Haskel, but I'm not getting any documents about it. How can I install a drain flow which reads lines from a file and works in parallel (i.e. with 8 cores, it should read eight lines, and be handed over to eight different threads to process them, And then to be re-collected etc.), possibly as small as the "function" ...

Alternatively, it can be noted that the lines need to be added in sequence Have Or not, if it could affect the speed of the process?

I am sure that it would be possible to use ideas from parallel Haskell's book to spray them together, but it seems that uniform work should be performed in the middle of a drain workflow Too easy? Petr Pudlák, in his comment as an example of "internal similarity", consider the function (

I Pipes , but can be easily implemented with conduit ): 

  Import control. Mono import control. Lens (See) Import Control Concentrator. ACICNC import pipes import qualified pipe. Group G imports as a valid control. L concprod :: INT - & gt; (A -> Io B) - & gt; Manufacturer is an IoR - & gt; Producer BIO RConprod Groupis Action Producer = L. Peerly G. Faldus L. List (see (G. Chanchu Group Group) producer) & gt; - & gt; Forever (wait> gt; = .ioi map, expected action> gt; = mapM G.yield)

This function takes the form of criteria for a group size , A verb that we want to run for each a of value, and creator of a value

this one The new manufacturer returns internally, the manufacturer codifies a values ​​, in the norms of them; Processes the process, and produces the results one by one.

Use the code Pipes.Group to originally "split" the "sub-creator" size to groupize and then each sub-productive To "fold", Control.Foldl in a list.

For more sophisticated functions, you can convert to asynchronous channels provided by pipe-concurrency or stm-conduit . But you get out of the parts of the "single pipeline" worldview of vanilla pipes / rivers.


Comments

Post a Comment

Popular posts from this blog

HTML/CSS - Automatically set height width from background image? -

-
Long time ... I do not think it's possible, but I'm shocked before! I tag anchor tags, all of which are background images, all 300px wide but their height is all the difference, is it really set to do without typing altitude? Is it to set the dimensions of BG URL? Thank you! I do not think people understand - my mistake is to hurry the question. Here is the code as an example: # ago -1 {width: 300px; Height: 410px; Background: url ('/ image-1.jpg');} # East -2 {width: 300px; Height: 420px; Background: url ('/ image-2.jpg');} # Pre-3 {width: 300px; Height: 430px; Background: url ('/ image-3.jpg');} # Pre-4 {width: 300px; Height: 440px; Background: url ('/ image-3.jpg');} & lt; A href = "#" id = "ex-1" & gt; & Lt; / A & gt; & Lt; A href = "#" id = "ex-2" & gt; & Lt; / A & gt; & Lt; A href = "#" id = "pre-3" & gt; & Lt; / A & gt...
Read more

php - Mysql Show Process - Sleep Commands and what to do -

-
I am thinking that there is something that I can do better so that my process list should start with the command with MySQL Be clear Currently I am looking for a lot: 17325 user_a localhost db_1 sleep 1132 NULL 17464 user_a localhost db_1 sleep 1124 NULL 17983 user_a localhost db_1 sleep 1078 null 18113 user_a localhost db_1 sleep 1068 NULL 18207 user_a localhost db_1 sleep 1060 NULL 18231 user_a localhost db_1 sleep 1058 NULL 18353 user_a localhost db_1 sleep 1047 NULL 18447 user_a localhost db_1 sleep 1040 null 18489 user_a localhost db_1 sleep 1036 NULL 23408 user_a localhost db_1 sleep 637 NULL Something in my phpscript is that I want to clean it Can i The script looks like: Select the ID from $ q = 'db_1 where fkId = 2'; $ Ar = mysql_query ($ q); ($ Line = mysql_fetch_array ($ qr)) {echo ($ row ['id'];} Thanks in advance! I'm guessing that when you have 10 threads, you're constantly using the connectio...
Read more

c - What is the address of buf (the local variable in the main function)? -

-
I had exercise with programming in c or classes and I followed the question: Is Buf (local variable in main function)? Either enter the answer in hexadecimal format (like 0x 8 "digits" 0-9 or A-F, 0xbfff0014) or in decimal format. Note here that we want the address of the buff, not its content. The code looks like this: int main (int argc, char * argv []) {while (1) {four buff [1024] = {0}; Int r; .... I then started the code and put the built code in the buff [1024] line and in Typed Gdb p & amp; Buf and it gives me results: 0xbffff0f0 But when I use it in this quiz it gives me the result that this value is wrong. My question is this: Is this the name of the Buff variable (P & E)? Or if not, why not? Exercise is done on the virtualbox machine, so I think everyone should be the same address There is no correct numerical answer, the address may vary from one time to another, it is a programming drill that you say the following three...
Read more