Parallel.For with/update List example, ThreadLocal Explained
FINAL CODE:
Parallel.For(0, lCount, () => new List<int>(), (i, loop, lLocal) =>
{
lLocal.Add(i); return lLocal;
},
(lFinal) =>
{ lock (this)
list = list.Union(lFinal.AsEnumerable()).ToList();
}
);
EXPLANATION:
did u read MS example and had a hard time? me 2 :)
http://msdn.microsoft.com/en-us/library/dd460703.aspx
so lets make it simpler:
1 - what we're doing is this
for (int i = 0; i < nums.Length; i++)
total += (long)i;
2 - the catch: http://msdn.microsoft.com/en-us/library/dd997392.aspx
3 - the solution - give the loop(s - i'll show u later) a local var to use during the iteration
4 - whats the new loop is doing :
Parallel.For<long>( // <T> is the type that we are working with, here
summing a long
0, nums.Length, // begin-end iteration values,
i.e. "int i = 0; i < nums.Length;"
() => 0, //initialize your local state (var), out subtotal
(j, loop, subtotal) =>
//is something like this public <T> DoWork(int i,
ParallelLoopState, <T>)
//int i is the current i value in the loop
//ParallelLoopState - instance to break the loop.
//<T> - our working var, each DoWork gets it,
work on it, and return its like (or it)
{
//since i am a lambda created, u dont need do new to my inputs,
nor declare my return
subtotal += nums[j];
return subtotal;
},
(x) => Interlocked.Add(ref total, x) // type <T>
// will be called after a set(!) of iterations completed.
);
great!
but now i wanna do this
for (int i = 0; i < 10000; i++)
list.Add(i);
which becomes this
Parallel.For(0, 100000, () => new List<int>(),// we want to work on filling a list
(i, loop, lLocal) => // lLocal is our list, our local var
{
lLocal.Add(i);
return lLocal; // and therefor our return type
},
(lFinal) => Interlocked.Exchange<List<int>>(ref list, lFinal);
// i.e. put lFinal in list, could also be just list = lFinal
);
i got about half the numbers!?!
if u read the catch part u'll see they're talking about it.
lets put in the MS example this:
(lFinal) =>
{
Interlocked.Exchange<List<int>>(ref l, lFinal);
Console.WriteLine(lFinal.Count);
}
our output will be something like:
XXXX // like 2856
XXXX
XXXX
XXXX
usually between 4-6.
meaning Parallel does 4-6 threads, each updating list
so we need to create something thread safe AND unioning to our list
Interlocked is a special class that is making the exchange "atomic", as in atomic speed, so its what we want to lock "real time" something like our update to our list
so i got to this:
int usingList = 0;
Parallel.For(0, lCount, () => new List<int>(), (i, loop, lLocal) =>
{
lLocal.Add(i);
return lLocal;
},
(lFinal) =>
{
while (true)
{
if (Interlocked.Exchange(ref usingList, 1) == 0)
{
list = list.Union(lFinal.AsEnumerable()).ToList();
Interlocked.Exchange(ref usingList, 0);
break;
}
else
Thread.Sleep(3);
}
Console.WriteLine("done " + lFinal.Count);
});
luckly .NET have something that does this exactly
FINAL CODE:
Parallel.For(0, lCount, () => new List<int>(), (i, loop, lLocal) =>
{
lLocal.Add(i);
return lLocal;
},
(lFinal) =>
{
lock (this)
list = list.Union(lFinal.AsEnumerable()).ToList();
}
);
P.S. - for this simple task i found out that this also works
Parallel.For(0, lCount, (int i) =>
{
lock (this) l.Add(i);
});
Parallel.For(0, lCount, () => new List<int>(), (i, loop, lLocal) =>
{
lLocal.Add(i); return lLocal;
},
(lFinal) =>
{ lock (this)
list = list.Union(lFinal.AsEnumerable()).ToList();
}
);
EXPLANATION:
did u read MS example and had a hard time? me 2 :)
http://msdn.microsoft.com/en-us/library/dd460703.aspx
so lets make it simpler:
1 - what we're doing is this
for (int i = 0; i < nums.Length; i++)
total += (long)i;
2 - the catch: http://msdn.microsoft.com/en-us/library/dd997392.aspx
3 - the solution - give the loop(s - i'll show u later) a local var to use during the iteration
4 - whats the new loop is doing :
Parallel.For<long>( // <T> is the type that we are working with, here
summing a long
0, nums.Length, // begin-end iteration values,
i.e. "int i = 0; i < nums.Length;"
() => 0, //initialize your local state (var), out subtotal
(j, loop, subtotal) =>
//is something like this public <T> DoWork(int i,
ParallelLoopState, <T>)
//int i is the current i value in the loop
//ParallelLoopState - instance to break the loop.
//<T> - our working var, each DoWork gets it,
work on it, and return its like (or it)
{
//since i am a lambda created, u dont need do new to my inputs,
nor declare my return
subtotal += nums[j];
return subtotal;
},
(x) => Interlocked.Add(ref total, x) // type <T>
// will be called after a set(!) of iterations completed.
);
great!
but now i wanna do this
for (int i = 0; i < 10000; i++)
list.Add(i);
which becomes this
Parallel.For(0, 100000, () => new List<int>(),// we want to work on filling a list
(i, loop, lLocal) => // lLocal is our list, our local var
{
lLocal.Add(i);
return lLocal; // and therefor our return type
},
(lFinal) => Interlocked.Exchange<List<int>>(ref list, lFinal);
// i.e. put lFinal in list, could also be just list = lFinal
);
i got about half the numbers!?!
if u read the catch part u'll see they're talking about it.
lets put in the MS example this:
(lFinal) =>
{
Interlocked.Exchange<List<int>>(ref l, lFinal);
Console.WriteLine(lFinal.Count);
}
our output will be something like:
XXXX // like 2856
XXXX
XXXX
XXXX
usually between 4-6.
meaning Parallel does 4-6 threads, each updating list
so we need to create something thread safe AND unioning to our list
Interlocked is a special class that is making the exchange "atomic", as in atomic speed, so its what we want to lock "real time" something like our update to our list
so i got to this:
int usingList = 0;
Parallel.For(0, lCount, () => new List<int>(), (i, loop, lLocal) =>
{
lLocal.Add(i);
return lLocal;
},
(lFinal) =>
{
while (true)
{
if (Interlocked.Exchange(ref usingList, 1) == 0)
{
list = list.Union(lFinal.AsEnumerable()).ToList();
Interlocked.Exchange(ref usingList, 0);
break;
}
else
Thread.Sleep(3);
}
Console.WriteLine("done " + lFinal.Count);
});
luckly .NET have something that does this exactly
FINAL CODE:
Parallel.For(0, lCount, () => new List<int>(), (i, loop, lLocal) =>
{
lLocal.Add(i);
return lLocal;
},
(lFinal) =>
{
lock (this)
list = list.Union(lFinal.AsEnumerable()).ToList();
}
);
P.S. - for this simple task i found out that this also works
Parallel.For(0, lCount, (int i) =>
{
lock (this) l.Add(i);
});
Comments
Post a Comment