由买买提看人间百态

boards

本页内容为未名空间相应帖子的节选和存档,一周内的贴子最多显示50字,超过一周显示500字 访问原贴
Statistics版 - Problem with Maximum Likelihood Estimation
相关主题
用EM求的MLE,但是observed likelihood 怎么算呢any one did EM to find MLE of mixed effects model in R
likelihood ratio asymptotic approximationslog-likelihood
问个问题哈,关于maximum likelihood estimator问一个关于linear regression的error假设问题
Maximum Likelihood estimation[合集] question about MLE
MLE 问题againcalculate mle in sas (help)
maximum likelihood estimation请教高手 gaussian统计知识一问
Maximum Likelihood Estimator还是MLE分布拟合问题
在线急等:做maximum likelihood estimation,用optimization怎么都的不出正确的数值求复杂的likelihood的MLE除了EM还有什么方法
相关话题的讨论汇总
话题: theta话题: mu话题: sigma话题: likelihood话题: problem
进入Statistics版参与讨论
1 (共1页)
n**f
发帖数: 121
1
I am having great trouble with using Maximum Likelihood to estimate
distribution parameters. I will appreciate if anyone can help me find out
what went wrong.
Assume that I have M iid samples of random vector V, denoted by v_1,v_2,...v
_M. Define random variable X = f(V|theta) where f is function whose closed
form is known but parameter theta is unknown.
I assume X=f(V|theta) follows a lognormal distribution logN(mu,sigma) where
mu and sigma are unknown. My purpose is to JOINTLY estimate theta, mu, and
sigma. In practice all data samples come from simulation so Lognormal
distribution can be guaranteed.
I take the following approach: let us use g(x|mu,sigma) to denote the pdf
for logN(mu,sigma). The likelihood function L is then g(f(V|theta)|mu,sigma)
. Based on MLE, we then to maximize the sum of log(g(f(v_m|theta)|mu,sigma))
of all M samples. Since f(.) and g(.) are both known, this maximization problem can be
solved by nonlinear programming (at least in theory). The optimal solution
would provide the best estimates for theta, mu, and sigma simultaneously and
decent properties such as consistency follows.
In reality, however, when I inspect the result from nonlinear programming
solver, I routinely find that the estimates for theta, mu, and sigma are far
from the true values. I use the word true value because I am using
simulation data. In particular, in many case mu and sigma would fall on
boundary of the nonlinear programming problem. This happens even with large
numbers of data samples, i.e., when M = 10000. To make things worse, I
compare the value of likelihood function calculated at theory optimum (true
value) and optimization output. More often than not the a greater likelihood
(for sample data) is achieved at the optimization output, but not the true
value.
My question is, am I using MLE in the right way? I would appreciate if
anyone can point out the above description violates assumptions about MLE.
Many many thanks!
F****n
发帖数: 3271
2
MLE needs a pdf. In your case, g(X) is pdf, but you are actually estimating
g(X(V)), which is a new function of V. Make sure it is also a pdf of V.

.v
where

【在 n**f 的大作中提到】
: I am having great trouble with using Maximum Likelihood to estimate
: distribution parameters. I will appreciate if anyone can help me find out
: what went wrong.
: Assume that I have M iid samples of random vector V, denoted by v_1,v_2,...v
: _M. Define random variable X = f(V|theta) where f is function whose closed
: form is known but parameter theta is unknown.
: I assume X=f(V|theta) follows a lognormal distribution logN(mu,sigma) where
: mu and sigma are unknown. My purpose is to JOINTLY estimate theta, mu, and
: sigma. In practice all data samples come from simulation so Lognormal
: distribution can be guaranteed.

n**f
发帖数: 121
3
Thank you very much for the reply. I guess this is the problem.
However, the distribution of V is very hard to derive. Is there anyway to
get around that? Like method of moments?
Again many thanks!

estimating

【在 F****n 的大作中提到】
: MLE needs a pdf. In your case, g(X) is pdf, but you are actually estimating
: g(X(V)), which is a new function of V. Make sure it is also a pdf of V.
:
: .v
: where

o****o
发帖数: 8077
4
to me seems it falls naturally into Bayesian hierarchical framework
Have u tried Bayesian approach? Is the conditional distribution of [V_i|data, other parameters] hard to derive?

【在 n**f 的大作中提到】
: Thank you very much for the reply. I guess this is the problem.
: However, the distribution of V is very hard to derive. Is there anyway to
: get around that? Like method of moments?
: Again many thanks!
:
: estimating

n**f
发帖数: 121
5
V_i is the data I observe. But the close form distribution might be hard to
derive.

data, other parameters] hard to derive?

【在 o****o 的大作中提到】
: to me seems it falls naturally into Bayesian hierarchical framework
: Have u tried Bayesian approach? Is the conditional distribution of [V_i|data, other parameters] hard to derive?

F****n
发帖数: 3271
6
What do you mean by very hard to derive?
You cannot get the integral of g(X)?

【在 n**f 的大作中提到】
: Thank you very much for the reply. I guess this is the problem.
: However, the distribution of V is very hard to derive. Is there anyway to
: get around that? Like method of moments?
: Again many thanks!
:
: estimating

n**f
发帖数: 121
7
Thank you again for your reply.
g(X) is just standard LogNormal distribution. This part is straightforward.
My challenge is that: X is a scalar and V is a vector. So finding V=f'(x|
theta)
is very difficult -- at least for me. In reality, V is a K-dimensionional
vector that satisfies the following equation:
X = v_1 / sum_{i=1...K-1}{theta_k*v_k}.
One can argue that V is not identifiable with X and theta given.

【在 F****n 的大作中提到】
: What do you mean by very hard to derive?
: You cannot get the integral of g(X)?

F****n
发帖数: 3271
8
Are you sure you have a solution? If V is not identifiable with X and theta,
then there are potentially a lot of different Vs and thetas that can
produce the same X (quite true in your sum/aggregate X function). Even you
have a pre-defined data, there can be other parameter values that can
produce your pre-defined V.
IMO, you probably don't have an optimal solution no matter using what method.

【在 n**f 的大作中提到】
: Thank you again for your reply.
: g(X) is just standard LogNormal distribution. This part is straightforward.
: My challenge is that: X is a scalar and V is a vector. So finding V=f'(x|
: theta)
: is very difficult -- at least for me. In reality, V is a K-dimensionional
: vector that satisfies the following equation:
: X = v_1 / sum_{i=1...K-1}{theta_k*v_k}.
: One can argue that V is not identifiable with X and theta given.

n**f
发帖数: 121
9
Thank you for your patients comments. Now I know the issue is with modeling
rather than estimation.

theta,
method.

【在 F****n 的大作中提到】
: Are you sure you have a solution? If V is not identifiable with X and theta,
: then there are potentially a lot of different Vs and thetas that can
: produce the same X (quite true in your sum/aggregate X function). Even you
: have a pre-defined data, there can be other parameter values that can
: produce your pre-defined V.
: IMO, you probably don't have an optimal solution no matter using what method.

1 (共1页)
进入Statistics版参与讨论
相关主题
求复杂的likelihood的MLE除了EM还有什么方法MLE 问题again
One question about linear regression for interval censored datamaximum likelihood estimation
大家都是用什么软件estimate的?Maximum Likelihood Estimator
Regression中噪音项是一个AR(1),如何做MLE或者其它Fit?在线急等:做maximum likelihood estimation,用optimization怎么都的不出正确的数值
用EM求的MLE,但是observed likelihood 怎么算呢any one did EM to find MLE of mixed effects model in R
likelihood ratio asymptotic approximationslog-likelihood
问个问题哈,关于maximum likelihood estimator问一个关于linear regression的error假设问题
Maximum Likelihood estimation[合集] question about MLE
相关话题的讨论汇总
话题: theta话题: mu话题: sigma话题: likelihood话题: problem