w*s
发帖数: 7227 | 1 case 1,
linux system has 20 processes in total.
my app is 1 of that.
This app uses multi-thread, contains 2 threads.
case 2,
my app uses multi-process, so total 21 processes in the system.
my question is, what's the percentage of time slices my thread/process is ?
can i say,
for case 1, my process gets 1/20 of the cpu time, each thread gets 1/40 then.
for case 2, each of my app process gets 1/21 ?
(this is to simplify things, assuming same priority)
if so, the performance has a difference ?
many thanks ! | s********k
发帖数: 6180 | 2 两者都不是，这个问题应该不是一言两语能说清楚的，你看看最新的linux schedule吧
then.
【在 w*s 的大作中提到】
: case 1,
: linux system has 20 processes in total.
: my app is 1 of that.
: This app uses multi-thread, contains 2 threads.
: case 2,
: my app uses multi-process, so total 21 processes in the system.
: my question is, what's the percentage of time slices my thread/process is ?
: can i say,
: for case 1, my process gets 1/20 of the cpu time, each thread gets 1/40 then.
: for case 2, each of my app process gets 1/21 ?
| w*s
发帖数: 7227 | 3 大牛，请简单解释一下吧，等会上美图！
【在 s********k 的大作中提到】
: 两者都不是，这个问题应该不是一言两语能说清楚的，你看看最新的linux schedule吧
:
: then.
| s********k
发帖数: 6180 | 4 wiki下 Completely Fair Scheduler (CFS)，或者找本kernel development的书来看
，话说你为什么关心你自己的app运行时间？你不是开发kernel的这个基本没法控制？
难道你必要要实现real time？。而且还取决于看你程序是CPU bound还是IO bound了
【在 w*s 的大作中提到】
: 大牛，请简单解释一下吧，等会上美图！
| w*s
发帖数: 7227 | 5 太深刻了，看不懂。我的是CPU bound，主要想知道2个方法哪个效率高点。
上毛图。
【在 s********k 的大作中提到】
: wiki下 Completely Fair Scheduler (CFS)，或者找本kernel development的书来看
: ，话说你为什么关心你自己的app运行时间？你不是开发kernel的这个基本没法控制？
: 难道你必要要实现real time？。而且还取决于看你程序是CPU bound还是IO bound了
| s*******8
发帖数: 12734 | 6 是不是你想问同样的情况下, multi-thread vs multi-process?
mozilla firefox vs chrome?
哪个更快一些?
then.
【在 w*s 的大作中提到】
: case 1,
: linux system has 20 processes in total.
: my app is 1 of that.
: This app uses multi-thread, contains 2 threads.
: case 2,
: my app uses multi-process, so total 21 processes in the system.
: my question is, what's the percentage of time slices my thread/process is ?
: can i say,
: for case 1, my process gets 1/20 of the cpu time, each thread gets 1/40 then.
: for case 2, each of my app process gets 1/21 ?
| w*s
发帖数: 7227 | 7 有点点类似，但你问题的答案应该是multi-thread吧，重开一个browser process的
cost很大。
我的app 没这么expensive, 而且最多就3个process。
【在 s*******8 的大作中提到】
: 是不是你想问同样的情况下, multi-thread vs multi-process?
: mozilla firefox vs chrome?
: 哪个更快一些?
:
: then.
| s********k
发帖数: 6180 | 8 现代OS的scheduling都是基于thread的，最好的办法是你直接做实验
【在 w*s 的大作中提到】
: 有点点类似，但你问题的答案应该是multi-thread吧，重开一个browser process的
: cost很大。
: 我的app 没这么expensive, 而且最多就3个process。
| S*A
发帖数: 7142 | 9 你的问题本来就问得不好。
这个是没有确切回答得，因为条件并不足。你不知道
你得 process 和系统其他 process 有多忙。
case 1: 这个完全取绝与你那个 process 和 另外 process
在干什么活。系统得process 绝大多数时候都是 idle。
scheduler 并不需要运行你得 process。 但是你得 process
依照有多少个 thread， 如果知道一个，只能一次在一个 core
上运行。 如果你得机器有 8 个 core，你只能用到全部 CPU
资源得 1/8
case 2： 21 process 大概可以使用全部 CPU core， 如果都
忙得话。但是其实不必要。一个 process 用到 CPU core 个数
的 thread 就可以了。Thread 之间通信还容易，共享地址空间。 | s********k
发帖数: 6180 | 10 一个process的多个thread会有可能被分配到不同core上面去吗？这样的话swap之类岂
不是消耗很大？
【在 S*A 的大作中提到】
: 你的问题本来就问得不好。
: 这个是没有确切回答得，因为条件并不足。你不知道
: 你得 process 和系统其他 process 有多忙。
: case 1: 这个完全取绝与你那个 process 和 另外 process
: 在干什么活。系统得process 绝大多数时候都是 idle。
: scheduler 并不需要运行你得 process。 但是你得 process
: 依照有多少个 thread， 如果知道一个，只能一次在一个 core
: 上运行。 如果你得机器有 8 个 core，你只能用到全部 CPU
: 资源得 1/8
: case 2： 21 process 大概可以使用全部 CPU core， 如果都
| | | g*********e
发帖数: 14401 | 11 my experience is multi process will be faster. especially when you have sync
in thread scenario.
also depend on what is your process initializstion cost. say you process has
to load big files as libraries in the beginning , it would be slow to do it
several times. | g*********e
发帖数: 14401 | 12 of course if you have multi core cpus.
swap is just contrxt switch, sync/lock could be expensive, depend on the
program.
【在 s********k 的大作中提到】
: 一个process的多个thread会有可能被分配到不同core上面去吗？这样的话swap之类岂
: 不是消耗很大？
| w*s
发帖数: 7227 | 13 2nd this, 大牛，等等我详细描述一下。
sync
has
it
【在 g*********e 的大作中提到】
: my experience is multi process will be faster. especially when you have sync
: in thread scenario.
: also depend on what is your process initializstion cost. say you process has
: to load big files as libraries in the beginning , it would be slow to do it
: several times.
| w*s
发帖数: 7227 | 14 各位大牛，容我尽力描述一下，请大家畅所欲言，非常感谢。
linux system, two rs485 ports, ~50 devices attached on these 2 ports.
~100 processes in the system from linux itself. 4 core arm.
in many sense it's like a linux desktop with our app running.
could get very busy, e.g. like the moment you open office in linux.
1. original design was like this, also tested well enough,
The main app has 10 threads on its own.
For each rs485, main process spawns off a process to monitoring all
devices on this
port. (Since this is serial bus, it's basically a for loop doing tx/rx to
each device 1 by 1. We don't have locks inside the two rs485 processes.)
so total 3 processes.
Each rs485 process doesn't talk with each other, they only talk with main
process through named pipe.
2. now after all the linux architects left, ppl asks me to change to use
threads. So only 1 process, 12 threads.
With that rs485 thread crashes without clue, no core file.
Ppl looked at code again and again and still clueless.
They don't want to go back to process, since
1. 3 processes with same name is confusing.
2. we don't know how to attach gdb to the process to debug. If only 1
process, it's easier.
3. politics
If you guys can help me with these
Q1: i think we can change process name, right ?
or create pid files for each process, i'm new to these.
Q2: if you have 3 processes with same name, how hard is it for debugging ?
Especially i want to debug the child process,
how to do this in gdb ?
can i set breakpoint in the code saying this is child, after the fork() ?
Q3: why a thread can die without core file, while main process/thread can
always generate code file ?
Q4：what can you gain if you use threads instead of processes in this case ?
(in my opinion, only gains trouble.)
Q5: named pipe through threads or processes, performance big difference ?
Many many thanks !
【在 S*A 的大作中提到】
: 你的问题本来就问得不好。
: 这个是没有确切回答得，因为条件并不足。你不知道
: 你得 process 和系统其他 process 有多忙。
: case 1: 这个完全取绝与你那个 process 和 另外 process
: 在干什么活。系统得process 绝大多数时候都是 idle。
: scheduler 并不需要运行你得 process。 但是你得 process
: 依照有多少个 thread， 如果知道一个，只能一次在一个 core
: 上运行。 如果你得机器有 8 个 core，你只能用到全部 CPU
: 资源得 1/8
: case 2： 21 process 大概可以使用全部 CPU core， 如果都
| S*A
发帖数: 7142 | 15 CPU schedule 关 swapping 什么事情。swapping 是
管理虚拟地址空间的内存的。
多个thread会被分配到不同的 core， 如果 core 没事情干，
你有 thread running， 就会去跑。如果你只有一个 thread
running， 不同 core 会去抢。一个成功抢到了就其他的就
干等了。
【在 s********k 的大作中提到】
: 一个process的多个thread会有可能被分配到不同core上面去吗？这样的话swap之类岂
: 不是消耗很大？
| w*s
发帖数: 7227 | 16 "多个thread会被分配到不同的 core", share data会不会不方便？
【在 S*A 的大作中提到】
: CPU schedule 关 swapping 什么事情。swapping 是
: 管理虚拟地址空间的内存的。
: 多个thread会被分配到不同的 core， 如果 core 没事情干，
: 你有 thread running， 就会去跑。如果你只有一个 thread
: running， 不同 core 会去抢。一个成功抢到了就其他的就
: 干等了。
| S*A
发帖数: 7142 | 17 1） of course you can change process name
2) You can use gdb -p to debug the child process.
That is trivial.
3) You can insert printf inside the thread. The thread are
likely exited normally. BTW, thread can generate core file
just like process does.
4) More control and a little fewer memory. Actually
you don't need 3 thread to do this. You should be able
to do it in one thread/process using async IO. Poll/select
etc. There is only two file to select from.
5) It depends on how many IO transfer through the pipe.
How do you read it etc.
When your 3 process is running, how much CPU idle left
for the system? If none of the CPU core is close to 100%.
I then CPU performance is not the problem.
【在 w*s 的大作中提到】
: 各位大牛，容我尽力描述一下，请大家畅所欲言，非常感谢。
: linux system, two rs485 ports, ~50 devices attached on these 2 ports.
: ~100 processes in the system from linux itself. 4 core arm.
: in many sense it's like a linux desktop with our app running.
: could get very busy, e.g. like the moment you open office in linux.
: 1. original design was like this, also tested well enough,
: The main app has 10 threads on its own.
: For each rs485, main process spawns off a process to monitoring all
: devices on this
: port. (Since this is serial bus, it's basically a for loop doing tx/rx to
| S*A
发帖数: 7142 | 18 Nope, threads share the address space.
Share data is easier. Of course you need some multithreading programming.
How to access shared data etc
【在 w*s 的大作中提到】
: "多个thread会被分配到不同的 core", share data会不会不方便？
| w*s
发帖数: 7227 | 19 先谢谢大牛！
3) we are in the end of release, it could take days to reproduce,
printf is slow for debugging.
That rs485 threads are in a forever loop, NEVER exit unless error,
but i don't see any error in the console.
There's a kernel oops complaining about "bad PC value",
but no one in our team knows how to read that info without core.
4) we don't worry about memory, not quite understanding the "more control"
part, more pls !
The reason we have 3 proceses/threads is may just connect 1 rs485 cable.
The main thread is dedicated to handle a lot of other things.
So we separate things out.
5) not a lot IO, basically just read/write to the rs485 port.
Since it's serial bus, it touches 1 device a time, no competing.
6) For process case, cpu is fine. Actually for thread case, that thread died
in the midnight, which means no real data transaction, other than my app
pings the devices in the forever loop. So at that time cpu is idle.
Initially i was worried about thread won't be stable as process,
also was afraid that threads can step on each other.
But everyone else barked on me.
【在 S*A 的大作中提到】
: 1） of course you can change process name
: 2) You can use gdb -p to debug the child process.
: That is trivial.
: 3) You can insert printf inside the thread. The thread are
: likely exited normally. BTW, thread can generate core file
: just like process does.
: 4) More control and a little fewer memory. Actually
: you don't need 3 thread to do this. You should be able
: to do it in one thread/process using async IO. Poll/select
: etc. There is only two file to select from.
| w*s
发帖数: 7227 | 20 how's that different if i create shared memory for 2 processes ?
i thought if you have 2 threads in the same core, use core local memory,
this is good.
if you have threads in 2 cores, physically how do they access memory ?
that cannot be either core's local memory, right ?
【在 S*A 的大作中提到】
: Nope, threads share the address space.
: Share data is easier. Of course you need some multithreading programming.
: How to access shared data etc
| | | g*********e
发帖数: 14401 | 21
cores don't have local memory. they may have their own L1 cache. so you can
treat them as accessing the same main memory, or same address space of your
process. That's why there is race condition.
【在 w*s 的大作中提到】
: how's that different if i create shared memory for 2 processes ?
: i thought if you have 2 threads in the same core, use core local memory,
: this is good.
: if you have threads in 2 cores, physically how do they access memory ?
: that cannot be either core's local memory, right ?
| w*s
发帖数: 7227 | 22 i c, thanks !
can
your
【在 g*********e 的大作中提到】
:
: cores don't have local memory. they may have their own L1 cache. so you can
: treat them as accessing the same main memory, or same address space of your
: process. That's why there is race condition.
| S*A
发帖数: 7142 | 23 3) kernel oops explain you can't get a coredump.
The coredump happen when the kernel receive a signal like
segfault. If your process is too mess up, e.g. the BAD PC
means your EIP is invalid value, most likely mess up your
stack, then there is no way the process can do core dump.
Just forget coredump, get back to the debug logging. If you
don't have debug logging, then you should add that to your
code. Yes, adding debug logging is a bit slow, but how is
using gdb any faster if you can't make any progress.
Using debugger like gdb is the wrong way to debug long
running process like yours. Good luck with that.
4) Hmm, go do a little more research on the threading
model vs processing model. It is not some thing I can
write up in a post.
5) 6) if performance is not the problem, your existing code
works fine. You seems have a lot of problem trying to
migrate to the thread model. Why change it then?
【在 w*s 的大作中提到】
: 先谢谢大牛！
: 3) we are in the end of release, it could take days to reproduce,
: printf is slow for debugging.
: That rs485 threads are in a forever loop, NEVER exit unless error,
: but i don't see any error in the console.
: There's a kernel oops complaining about "bad PC value",
: but no one in our team knows how to read that info without core.
: 4) we don't worry about memory, not quite understanding the "more control"
: part, more pls !
: The reason we have 3 proceses/threads is may just connect 1 rs485 cable.
| w*s
发帖数: 7227 | 24 Thanks for the reply,
why we are changing to thread ?
very simple, 将帅无能，累死三军，最怕的就是这种半吊子水平的乱指挥。
等下上图。
【在 S*A 的大作中提到】
: 3) kernel oops explain you can't get a coredump.
: The coredump happen when the kernel receive a signal like
: segfault. If your process is too mess up, e.g. the BAD PC
: means your EIP is invalid value, most likely mess up your
: stack, then there is no way the process can do core dump.
: Just forget coredump, get back to the debug logging. If you
: don't have debug logging, then you should add that to your
: code. Yes, adding debug logging is a bit slow, but how is
: using gdb any faster if you can't make any progress.
: Using debugger like gdb is the wrong way to debug long
| w*s
发帖数: 7227 | | w*s
发帖数: 7227 | 26 CFS is based on tasks, so if i have 1 process, from linux it automatically
contains 1 thread, so this is counted as 1 task or 2 tasks (1 process + 1
thread) ?
see this post,
"if process A creates 100 threads and B creates 1 thread then the # of
running processes or threads becomes 103 (assuming all are runnable state)
then CFS will evenly share the cpu using formula 1/103 (cpu/number of
running tasks)."
how come he talked about 103 not 101 ?
http://stackoverflow.com/questions/21152325/in-linux-scheduler-
【在 s********k 的大作中提到】
: 现代OS的scheduling都是基于thread的，最好的办法是你直接做实验
| S*A
发帖数: 7142 | 27
No. If you have one process, it has one thread. In Linux kernel,
it is how many schedule-able struct call "task".
I think they mean, you have process A and B. There is 1 thread for
A and 1 for B.
A create another 100 threads. B create another 1.
So that is total 1 + 1 + 100 + 1 = 103
【在 w*s 的大作中提到】
: CFS is based on tasks, so if i have 1 process, from linux it automatically
: contains 1 thread, so this is counted as 1 task or 2 tasks (1 process + 1
: thread) ?
: see this post,
: "if process A creates 100 threads and B creates 1 thread then the # of
: running processes or threads becomes 103 (assuming all are runnable state)
: then CFS will evenly share the cpu using formula 1/103 (cpu/number of
: running tasks)."
: how come he talked about 103 not 101 ?
: http://stackoverflow.com/questions/21152325/in-linux-scheduler-
| w*s
发帖数: 7227 | 28 大牛，谢谢了。
也没啥好图了，请将就一下。
【在 S*A 的大作中提到】
:
: No. If you have one process, it has one thread. In Linux kernel,
: it is how many schedule-able struct call "task".
: I think they mean, you have process A and B. There is 1 thread for
: A and 1 for B.
: A create another 100 threads. B create another 1.
: So that is total 1 + 1 + 100 + 1 = 103
|
|
