[Openerp-community] odoo with a daily 50000 orders volume

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[Openerp-community] odoo with a daily 50000 orders volume

LIN Yu

Hello Community,


We have a potential lead which has a volume of daily 20000 sale orders and 50000 sale order lines from Magento, which will probably result in a daily 100000 or 200000 stock moves, 3000000 stock moves per month.

I would like to ask that has anyone experience on Odoo/OpenERP with that daily volume? How about the performance after one month?


I would like also to ask if you have any module on 

* archive stock move , sales order, and other objects

* sales forecasting


BR,


_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Openerp-community] odoo with a daily 50000 orders volume

Raphael Valyi
Hello Lin,

we worked indirectly on a case in the USA with 1/10 of the numbers you mentioned. And we had terrible issues. Just to be clear we were called as firemen and didn't had a chance to study things upfront on that project. Well basically, 2 things became extremely slow: workflows because tables were huge and stock moves (that was on 6.1) because millions of move to crawl to compute any stock level.

On v8, there is good hope that the stock scalability issue is gone. As for the workflows I'm afraid I'll probably still have issues: all workflows use the same tables and open workflow instances of sale, stock, account accumulate. My advice is: if it's on a version prior to v8, forget it because of the way the stock moves work. If it's on v8, well in any case plan some very serious optimizations and testing to see feasibility.

Number of concurrent users isn't too hard to tackle by adding more cores now that we have process based parallelism, but as for very large tables, well you'll have to clean them up often, you'll have tweak indexes, probably to bypass some computations and use some custom SQL, add caching...
I never heard about sharding in OpenERP/Odoo so I guess nobody ever did sharding, so don't assume it would be easy like in frameworks where it's being done...

As for the company we worked for, well, after 18 months they ended up dropping OpenERP for SAP. Probably putting way more money on it than trying again on v8 and paying a lot of engineering to get the things optimized, but still it's the decision they took and scalability was one of the main reasons and that was 1/10th of the figures you mentioned. So don't take that challenge lightly and good luck!

Regards.


-- 
Raphaël Valyi
Founder and consultant
+55 21 3942-2434




On Sun, Aug 24, 2014 at 10:21 PM, lin.yu <[hidden email]> wrote:

Hello Community,


We have a potential lead which has a volume of daily 20000 sale orders and 50000 sale order lines from Magento, which will probably result in a daily 100000 or 200000 stock moves, 3000000 stock moves per month.

I would like to ask that has anyone experience on Odoo/OpenERP with that daily volume? How about the performance after one month?


I would like also to ask if you have any module on 

* archive stock move , sales order, and other objects

* sales forecasting


BR,


_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp



_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Openerp-community] odoo with a daily 50000 orders volume

Raphael Valyi
When I said "clean tables", I should have been more explicit: if you crawl the bug tracker, you'll notice a good quantity of bugs with workflows that don't close properly when some conditions are meet. This also really depend on being organized. If you are not properly organized, workflows of business documents will stay open not even because of bugs but just because of incomplete processes.

In any case, it's critical to watch out the number of business documents with an open workflow as it's these that accumulate in the common workflow table and will make the workflow engine struggle in critical ways. So if you don't know what to test to get an idea of the feasibility, test this: create millions of business documents with thousands with an open workflow as it will definitely be one of the bottlenecks in your case.

Regards.

-- 
Raphaël Valyi
Founder and consultant
+55 21 3942-2434



On Sun, Aug 24, 2014 at 11:43 PM, Raphael Valyi <[hidden email]> wrote:
Hello Lin,

we worked indirectly on a case in the USA with 1/10 of the numbers you mentioned. And we had terrible issues. Just to be clear we were called as firemen and didn't had a chance to study things upfront on that project. Well basically, 2 things became extremely slow: workflows because tables were huge and stock moves (that was on 6.1) because millions of move to crawl to compute any stock level.

On v8, there is good hope that the stock scalability issue is gone. As for the workflows I'm afraid I'll probably still have issues: all workflows use the same tables and open workflow instances of sale, stock, account accumulate. My advice is: if it's on a version prior to v8, forget it because of the way the stock moves work. If it's on v8, well in any case plan some very serious optimizations and testing to see feasibility.

Number of concurrent users isn't too hard to tackle by adding more cores now that we have process based parallelism, but as for very large tables, well you'll have to clean them up often, you'll have tweak indexes, probably to bypass some computations and use some custom SQL, add caching...
I never heard about sharding in OpenERP/Odoo so I guess nobody ever did sharding, so don't assume it would be easy like in frameworks where it's being done...

As for the company we worked for, well, after 18 months they ended up dropping OpenERP for SAP. Probably putting way more money on it than trying again on v8 and paying a lot of engineering to get the things optimized, but still it's the decision they took and scalability was one of the main reasons and that was 1/10th of the figures you mentioned. So don't take that challenge lightly and good luck!

Regards.


-- 
Raphaël Valyi
Founder and consultant
<a href="tel:%2B55%2021%203942-2434" value="+552139422434" target="_blank">+55 21 3942-2434




On Sun, Aug 24, 2014 at 10:21 PM, lin.yu <[hidden email]> wrote:

Hello Community,


We have a potential lead which has a volume of daily 20000 sale orders and 50000 sale order lines from Magento, which will probably result in a daily 100000 or 200000 stock moves, 3000000 stock moves per month.

I would like to ask that has anyone experience on Odoo/OpenERP with that daily volume? How about the performance after one month?


I would like also to ask if you have any module on 

* archive stock move , sales order, and other objects

* sales forecasting


BR,


_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp




_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Openerp-community] odoo with a daily 50000 orders volume

Marcelo Bello
In reply to this post by LIN Yu
About your last point - sales forecasting - recently there was a thread open on the list about Reorder Point automatic calculation (sales forecasting is what feeds Reorder Point information).

On that thread many solutions were proposed, it is worth checking. I mentioned about Lokad's Salescast, a cloud-based sales forecasting tool that I am paying a company to integrate with Odoo (module should be available before year-end but not because it is difficult to integrate, just because it is not on the top of my priorities). Have a look at lokad.com, check Salescast and if you are interested I can give you more details.

Best regards,

Marcelo

On Sun, Aug 24, 2014 at 10:21 PM, lin.yu <[hidden email]> wrote:

Hello Community,


We have a potential lead which has a volume of daily 20000 sale orders and 50000 sale order lines from Magento, which will probably result in a daily 100000 or 200000 stock moves, 3000000 stock moves per month.

I would like to ask that has anyone experience on Odoo/OpenERP with that daily volume? How about the performance after one month?


I would like also to ask if you have any module on 

* archive stock move , sales order, and other objects

* sales forecasting


BR,


_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp



_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Openerp-community] odoo with a daily 50000 orders volume

Alberto Barrionuevo
In reply to this post by LIN Yu
Hi Lin Yu,

     we have dealed with several thousand daily orders already in a big
logistic and retail installation for verscion 6.0 (including custom
e-commerce and SAP R/3 integrations, and ten of thousand products).

     Odoo/OpenERP will work but you will need:

a) clustering;

b) to do changes in the ORM of the Odoo system or, directly, to jump
over it to query directly to PostgreSQL, or, probably both; and, finally,

c) a kind of middleware that deals correctly with the communications
between the third party systems (Magento et al.) and the OpenERP.

     For the third point we developed a highly optimized on performance
middleware for OpenERP that helps taking care of a big part of the
workload and, so, demanding from OpenERP as less as possible.

     But the middleware is not enough, so I have to prevent you that you
will need an on demand solution for the concrete installation of your
customer. The most complex thing is dealing correctly with the huge
concurrency you will have in your optimized system.

     In versions 6.1 and 7.0 you will need less optimizations in the
Odoo core, but still some and they will still depend on your
customization (specially in warehouse -stocks calculation, sequence
generation, other locking objects,...-, sales orders and XML-RPC
communications).

     After all the optimizations to the Odoo core are done, your next
escalation limit, if you provide enough nodes in your cluster, will be
PostgreSQL, not OpenERP anymore. So your question should be if
PostgreSQL can deal with the volumes you need (and it should but
possibly will need also performance customizations).

     If you need I can forward you some videos showing our solution
(middleware and OpenERP) working in real time.

Best regards,
--
Alberto Barrionuevo, Director
OPENTIA Group
Odoo/OpenERP Partners
T (+34) 918 38 38 58
F (+34) 944 34 00 77

            OPENTIA
      Open IT for Open Minds
         www.opentia.com


On 25/08/14 03:21, lin.yu wrote:

> Hello Community,
>
>
> We have a potential lead which has a volume of daily 20000 sale orders
> and 50000 sale order lines from Magento, which will probably result in a
> daily 100000 or 200000 stock moves, 3000000 stock moves per month.
>
> I would like to ask that has anyone experience on Odoo/OpenERP with that
> daily volume? How about the performance after one month?
>
>
> I would like also to ask if you have any module on
>
> * archive stock move , sales order, and other objects
>
> * sales forecasting
>
>
> BR,
>
>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openerp-community
> Post to     : [hidden email]
> Unsubscribe : https://launchpad.net/~openerp-community
> More help   : https://help.launchpad.net/ListHelp
>



_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: [Openerp-community] odoo with a daily 50000 orders volume

Ferdinand Gassauer
In reply to this post by LIN Yu
On 2014-08-25 03:21, lin.yu wrote:

for accounting you may encounter a similar problem
a solution is to use and enhance (change functions and reports to use period sums instead of account_move_lines)
production http://bazaar.launchpad.net/~camptocamp/c2c-rd-addons/6.1/files/head:/chricar_account_period_sum/
partly production http://bazaar.launchpad.net/~camptocamp/c2c-rd-addons/7.0/files/head:/chricar_account_period_sum/
ported http://bazaar.launchpad.net/~camptocamp/c2c-rd-addons/8.0/files/head:/chricar_account_period_sum/

Hello Community,


We have a potential lead which has a volume of daily 20000 sale orders and 50000 sale order lines from Magento, which will probably result in a daily 100000 or 200000 stock moves, 3000000 stock moves per month.

I would like to ask that has anyone experience on Odoo/OpenERP with that daily volume? How about the performance after one month?


I would like also to ask if you have any module on 

* archive stock move , sales order, and other objects

* sales forecasting


BR,



_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp


-- 
Ferdinand Gassauer
Eigentümer
ChriCar Bet. u. Ber. GmbH

_______________________________________________
Mailing list: https://launchpad.net/~openerp-community
Post to     : [hidden email]
Unsubscribe : https://launchpad.net/~openerp-community
More help   : https://help.launchpad.net/ListHelp
Loading...