Re: [Openerp-community] OpenERP: Hardware sizing for OpenERP

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [Openerp-community] OpenERP: Hardware sizing for OpenERP

Olivier Dony (OpenERP)
Let's take this rather technical discussion about sizing and configuring
an OpenERP production environment to the community list...
See my random comments below in the text..


On 01/05/2012 08:38 PM, Rostam Azarbehi wrote:
> Hi Everyone,
>
> Just my two cents, if that is ok. I agree with the comments, there
> should be some basic expectations for each of the major components of
> OpenERP. The postgreSQL is pretty straightforward as that info is
> readily available. However, no clear measures are there for the OpenERP
> server, or the web-server.

PostgreSQL is probably one of the most important component to configure,
because the default settings are quite conservative and mean you might
not be getting the best performance out of your hardware.
This is the reason why I mentioned mostly postgres-related links [2,3,4]

The other OpenERP components (web and server) don't have configuration
settings that would make such a big "generic" difference. What matters
here is the deployment architecture: whether you will be running all
components on the same machine or not, whether you will be doing
load-balancing, etc.

OpenERP 6.1 introduces several important changes[5] to make the server
truly stateless and WSGI compliant, so that a really scalable deployment
architectures is easier to achieve (including harnessing all CPU cores
for OpenERP!)
We don't have any recommended settings yet, but you can find a sample
"gunicorn.conf.py" configuration file [6] in the server to get started.
All feedback on deployment experience with gunicorn, mod_wsgi or others
would be most welcome, and I'll be glad to summarize that in our
installation documentation for 6.1.

The server team also plans to improve soon (early 2012, next version)
the isolation of "OpenERP Workers" to make it possible to enforce
per-process limits. As part of this task we should be benchmarking
typical OpenERP flows using various configurations, and thus we'll be
able to provide some raw benchmark numbers.


> In other solutions, like General Dynamics and
> Prophet 21, we have received recommended requirements based on load
> factors for an installation with base components. Yes, there are many
> complexities in OpenERP, especially when you consider all the modules
> partners have created, but basic specs based on the base modules should
> be easy to create for different load factors. As partners we can factor
> in buffer capacity for additional modules and complexities when we start
> working with the clients.

I'd be interested to see in what form the "load factor"-based
requirements were provided to you, to see if similar numbers could be
provided for OpenERP (unless this is confidential).

For OpenERP 6.0 and earlier the "generic" requirements are so simple
compared to other solutions that they're barely worth mentioning:
  *one decent server machine*.
Who would be serious enough to ask about hardware requirements and still
expect to deploy on less than that anyway?
For more precise requirements, they depend on so many project-specific
factors that they'd quickly be misleading... If you expect heavy load
(e.g several dozens of active concurrent users), split the database and
OpenERP server on separate similar-spec'd machines. And this rule
remains the most important one:
   *test in real conditions and measure!*

Note: a decent server machine means a recent multi-core machine with a
lot of RAM (it's cheap, 16Gb min) and fast RAIDed disks.

As of 6.1 and with a WSGI-based deployment with multiple workers, you
should be able to get more bang for the buck on your existing
infrastructure, but we don't have numbers yet.


> On 12-01-04 02:35 PM, Hery Atmadja wrote:
>> Hi everyone,
>>
>> While I do share Olivier's viewpoint in principle, as Carlos has
>> kindly pointed out it is ultimately beneficial for OpenERP and
>> partners to move towards the direction of establishing basic
>> framework for hardware sizing. Unfortunately, the added complexity
>> in all of this also lies with the fact _not a single entity/party_
>> knows exactly how all the various modules (authored by many
>> parties) would impact the sizing (load).
>>
>> Lacking a more 'methodical' way of creating a 'yardstick', a
>> possible viable alternative is to start comparing it to live sites.
>> With enough data, potential 'methods' or 'sizing parameters' may
>> be surmised as a start. The option is either we stay put or we
>> move forward together.

Sure, and I'll be glad to summarize any feedback regarding performance
and deployment settings in our documentation.


>> On 01/05/2012 12:05 AM, Carlos Almeida wrote:
>>> Hello everyone,
>>>
>>> I found Olivier email very usefull, thank you.
>>>
>>> But your email points our attention only to postgres. The links
>>> are very usefull but they only refers about the database
>>> optimization.

As I mentioned, it is the main thing that needs configuration.


>>> There must be parametrisation and/or some configuration rule of
>>> thumb for OpenERP, for example postgres suggest 50MB of
>>> maintenance_work_mem for every GB of RAM, something like this.

The work-horse of OpenERP is PostgreSQL, so this is where such
configuration settings matter.


>>> Also in OpenERP we have to deal with (i) server and (ii) web
>>> client paramenters, it's even worst when they are separated,
>>> because they have to comunicate through sockets, and so on...

The configuration parameters of OpenERP won't affect the performance
except in very specific cases, they're mostly present to connect the
various components together.


>>> At least having some "starting base" to work on, or some basic
>>> (not so right) rules will be great for a start. I'm talking about
>>> memory settings, time outs and number of connections related
>>> ones (eg: server.thread_pool, server.socket_queue_size,
>>> server.socket_timeout, openerp.server.timeout, db_maxconn,
>>> etc).

The timeout settings start to matter if you're hitting them, which means
you're doing some long-running operations or you're already having
performance issues. They won't solve any perf issue, and fine-tuning
them is specific to the reason you're hitting them.

The main connection limit that matters is PostgreSQL's max_connections
(again), which should be bumped up according to the number of concurrent
users you expect to service: 10 concurrent requests = 10 db connections.
The maxconn limit of the server should be set accordingly depending on
the number of servers/workers that are connecting to the db.


[1]
http://wiki.postgresql.org/wiki/Replication,_Clustering,_and_Connection_Pooling
[2] pgtune utility: https://github.com/gregs1104/pgtune
[3] http://blip.tv/djangocon/secrets-of-postgresql-performance-5572403
[4] http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server
[5] See also draft 6.1 release notes http://bit.ly/zzxdsK
[6]
http://bazaar.launchpad.net/~openerp/openobject-server/trunk/view/head:/gunicorn.conf.py

_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp
Reply | Threaded
Open this post in threaded view
|

Re: [Openerp-community] OpenERP: Hardware sizing for OpenERP

Hery Atmadja
Hi Olivier,

My comment on the live references is stated below in the text...

On 01/06/2012 11:20 PM, Olivier Dony wrote:
Let's take this rather technical discussion about sizing and configuring
an OpenERP production environment to the community list...
See my random comments below in the text..


On 01/05/2012 08:38 PM, Rostam Azarbehi wrote:
Hi Everyone,

Just my two cents, if that is ok. I agree with the comments, there
should be some basic expectations for each of the major components of
OpenERP. The postgreSQL is pretty straightforward as that info is
readily available. However, no clear measures are there for the OpenERP
server, or the web-server.
PostgreSQL is probably one of the most important component to configure,
because the default settings are quite conservative and mean you might
not be getting the best performance out of your hardware.
This is the reason why I mentioned mostly postgres-related links [2,3,4]

The other OpenERP components (web and server) don't have configuration
settings that would make such a big "generic" difference. What matters
here is the deployment architecture: whether you will be running all
components on the same machine or not, whether you will be doing
load-balancing, etc.

OpenERP 6.1 introduces several important changes[5] to make the server
truly stateless and WSGI compliant, so that a really scalable deployment
architectures is easier to achieve (including harnessing all CPU cores
for OpenERP!)
We don't have any recommended settings yet, but you can find a sample
"gunicorn.conf.py" configuration file [6] in the server to get started.
All feedback on deployment experience with gunicorn, mod_wsgi or others
would be most welcome, and I'll be glad to summarize that in our
installation documentation for 6.1.

The server team also plans to improve soon (early 2012, next version)
the isolation of "OpenERP Workers" to make it possible to enforce
per-process limits. As part of this task we should be benchmarking
typical OpenERP flows using various configurations, and thus we'll be
able to provide some raw benchmark numbers.


In other solutions, like General Dynamics and
Prophet 21, we have received recommended requirements based on load
factors for an installation with base components. Yes, there are many
complexities in OpenERP, especially when you consider all the modules
partners have created, but basic specs based on the base modules should
be easy to create for different load factors. As partners we can factor
in buffer capacity for additional modules and complexities when we start
working with the clients.
I'd be interested to see in what form the "load factor"-based
requirements were provided to you, to see if similar numbers could be
provided for OpenERP (unless this is confidential).

For OpenERP 6.0 and earlier the "generic" requirements are so simple
compared to other solutions that they're barely worth mentioning:
  *one decent server machine*.
Who would be serious enough to ask about hardware requirements and still
expect to deploy on less than that anyway?
For more precise requirements, they depend on so many project-specific
factors that they'd quickly be misleading... If you expect heavy load
(e.g several dozens of active concurrent users), split the database and
OpenERP server on separate similar-spec'd machines. And this rule
remains the most important one:
   *test in real conditions and measure!*

Note: a decent server machine means a recent multi-core machine with a
lot of RAM (it's cheap, 16Gb min) and fast RAIDed disks.

As of 6.1 and with a WSGI-based deployment with multiple workers, you
should be able to get more bang for the buck on your existing
infrastructure, but we don't have numbers yet.


On 12-01-04 02:35 PM, Hery Atmadja wrote:
Hi everyone,

While I do share Olivier's viewpoint in principle, as Carlos has 
kindly pointed out it is ultimately beneficial for OpenERP and 
partners to move towards the direction of establishing basic
framework for hardware sizing. Unfortunately, the added complexity
in all of this also lies with the fact _not a single entity/party_
knows exactly how all the various modules (authored by many
parties) would impact the sizing (load).

Lacking a more 'methodical' way of creating a 'yardstick', a
possible viable alternative is to start comparing it to live sites.
With enough data, potential 'methods' or 'sizing parameters' may
be surmised as a start. The option is either we stay put or we
move forward together.
Sure, and I'll be glad to summarize any feedback regarding performance
and deployment settings in our documentation.
Moving forward, we, at Infinity, would be glad to provide OpenERP with the number using our references - however due to confidentiality nature, the client name shall have to be concealed. I do hope other partners can contribute as well.
On 01/05/2012 12:05 AM, Carlos Almeida wrote:
Hello everyone,

I found Olivier email very usefull, thank you.

But your email points our attention only to postgres. The links
are very usefull but they only refers about the database
optimization.
As I mentioned, it is the main thing that needs configuration.


There must be parametrisation and/or some configuration rule of
thumb for OpenERP, for example postgres suggest 50MB of 
maintenance_work_mem for every GB of RAM, something like this.
The work-horse of OpenERP is PostgreSQL, so this is where such
configuration settings matter.


Also in OpenERP we have to deal with (i) server and (ii) web
client paramenters, it's even worst when they are separated,
because they have to comunicate through sockets, and so on...
The configuration parameters of OpenERP won't affect the performance
except in very specific cases, they're mostly present to connect the
various components together.


At least having some "starting base" to work on, or some basic
(not so right) rules will be great for a start. I'm talking about
memory settings, time outs and number of connections related
ones (eg: server.thread_pool, server.socket_queue_size,
server.socket_timeout, openerp.server.timeout, db_maxconn,
etc).
The timeout settings start to matter if you're hitting them, which means
you're doing some long-running operations or you're already having
performance issues. They won't solve any perf issue, and fine-tuning
them is specific to the reason you're hitting them.

The main connection limit that matters is PostgreSQL's max_connections
(again), which should be bumped up according to the number of concurrent
users you expect to service: 10 concurrent requests = 10 db connections.
The maxconn limit of the server should be set accordingly depending on
the number of servers/workers that are connecting to the db.


[1]
http://wiki.postgresql.org/wiki/Replication,_Clustering,_and_Connection_Pooling
[2] pgtune utility: https://github.com/gregs1104/pgtune
[3] http://blip.tv/djangocon/secrets-of-postgresql-performance-5572403
[4] http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server
[5] See also draft 6.1 release notes http://bit.ly/zzxdsK
[6]
http://bazaar.launchpad.net/~openerp/openobject-server/trunk/view/head:/gunicorn.conf.py

--
Regards,
Hery Atmadja
i n f i n i t y . s o l u t i o n

PT. Vikasa Infinity Anugrah (www.infi-nity.com)
BSD City Sektor 14, Ruko Golden Madrid G/9, Tangerang Selatan 15321 - INDONESIA
t: +62 (21) 5316 4796 f: +62 (21) 5316 4797 m:+62 816 956207


This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.

_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp
Reply | Threaded
Open this post in threaded view
|

Re: [Openerp-community] OpenERP: Hardware sizing for OpenERP

Olivier Dony (OpenERP)
On 01/09/2012 10:40 AM, Hery Atmadja wrote:
> Moving forward, we, at Infinity, would be glad to provide OpenERP with
> the number using our references - however due to confidentiality nature,
> the client name shall have to be concealed. I do hope other partners can
> contribute as well.

Excellent! As a starting point we could use this google spreadsheet,
feel free to edit and to add columns that you think would be relevant:

https://docs.google.com/spreadsheet/ccc?key=0AuuV1YRJafDgdG9BNGpfSXZHNU5ncFhVWGJ6QkcwVGc

_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp
Reply | Threaded
Open this post in threaded view
|

Re: [Openerp-community] OpenERP: Hardware sizing for OpenERP

James Jesudason
In reply to this post by Olivier Dony (OpenERP)

On 06/01/12 16:20, Olivier Dony wrote:
> OpenERP 6.1 introduces several important changes[5] to make the server
> truly stateless and WSGI compliant, so that a really scalable deployment
> architectures is easier to achieve (including harnessing all CPU cores
> for OpenERP!)
> We don't have any recommended settings yet, but you can find a sample
> "gunicorn.conf.py" configuration file [6] in the server to get started.
> All feedback on deployment experience with gunicorn, mod_wsgi or others
> would be most welcome, and I'll be glad to summarize that in our
> installation documentation for 6.1.

I welcome the changes that have been made to make OpenERP WSGI compliant
and truly stateless. However, if anyone is planning to to deploy 6.1
using gunicorn with multiple workers they should be aware of some
important bugs:
  * Bug #954907: changing company when running with multiple workers in
a multi-company environment does not work correctly. Some of the workers
will give you data from the wrong company.
  * Bug #944273: the scheduler does not work due to the multiple
processes involved. A fix for this is in progress.
  * openid/SSO: if you use the auth_openid module, then you'll need to
use a filestore. There is a TODO in the code that mentions that this is
planned. I think that one of my team has a merge proposal for this on
her TODO list.

To be fair, I think that the release notes say the OpenERP 6.1 is now
*mostly* stateless... which is true!

James


_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp
Reply | Threaded
Open this post in threaded view
|

Re: [Openerp-community] OpenERP: Hardware sizing for OpenERP

Hery Atmadja
In reply to this post by Olivier Dony (OpenERP)
It's an old topic, but as promised we just added a reference entry.  Hope this helps.

On 01/09/2012 07:18 PM, Olivier Dony wrote:
On 01/09/2012 10:40 AM, Hery Atmadja wrote:
Moving forward, we, at Infinity, would be glad to provide OpenERP with
the number using our references - however due to confidentiality nature,
the client name shall have to be concealed. I do hope other partners can
contribute as well.
Excellent! As a starting point we could use this google spreadsheet,
feel free to edit and to add columns that you think would be relevant:

https://docs.google.com/spreadsheet/ccc?key=0AuuV1YRJafDgdG9BNGpfSXZHNU5ncFhVWGJ6QkcwVGc

_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp


--
Regards,
Hery Atmadja
i n f i n i t y . s o l u t i o n

PT. Vikasa Infinity Anugrah (www.infi-nity.com)
BSD City Sektor 14, Ruko Golden Madrid G/9, Tangerang Selatan 15321 - INDONESIA
t: +62 (21) 5316 4796 f: +62 (21) 5316 4797 m:+62 816 956207


This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.

_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp
Reply | Threaded
Open this post in threaded view
|

Re: [Openerp-community] OpenERP: Hardware sizing for OpenERP

Olivier Dony (OpenERP)
On 04/25/2012 08:22 AM, Hery Atmadja wrote:
> It's an old topic, but as promised we just added a reference entry.  Hope this
> helps.

Excellent, this is invaluable! It's always hard for partners to find the time
to publish references, especially wrt hardware sizing (as you can see, there is
not much in the spreadsheet yet). Often as you build experience you forget how
much you were looking for such information when you started...

We will soon add a "References" section in the Partner Portal, and maybe having
it directly in the portal will remind all Partners of the value of sharing them.

Thanks a lot!

_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp