Follow us on Twitter X-Cart on Facebook Wiki
Shopping cart software Solutions for online shops and malls

XC5 Import Routine Slowing

 
Reply
   X-Cart forums > X-Cart 5 > Dev Questions (X-Cart 5)
 
Thread Tools Search this Thread
  #1  
Old 01-08-2015, 01:07 PM
 
Mark N Mark N is offline
 

Senior Member
  
Join Date: Sep 2011
Posts: 121
 

Angry XC5 Import Routine Slowing

Testing out X-Cart 5 (Business version, 5.1.9) and am trying out the import routine. Needless to say, the results are not encouraging.

After getting past the hurdle that you cannot auto-create product classes by importing an attributes file that references them (it just locks up on import, no error given), I was able to do an initial import of categories and attributes.

Importing products, I started with a small subset of our total product inventory - 5000 SKUs. We have over 55,000 total so this is a pretty small number for us.

The import is running now, and the results aren't encouraging. Every 100 items imported takes longer than the last 100 did. For example, for the first 1000 items, it was taking about 10-15 seconds per 100. The second 1000, that number went up to close to 30 seconds per 100. Now, 22 minutes into the import, it is around item 4000, taking over a minute to import 100 items, with the time steadily increasing. If I were trying to import all 55,000 items it would never finish - by my calculations by the time it gets out to 50k+ items, each 100 items will take over 10 minutes to run.

Has anyone else seen this behavior? The fact that it is steadily increasing leads me to believe there is a resource leak somewhere.

I'm attaching a small subset csv, just to rule out an error in how I've set up the file.
Attached Files
File Type: csv products-2015-01-08-subset.csv (5.4 KB, 44 views)
__________________
X-Cart Gold Plus 4.6.5
Mods - WebsiteCM Dynamic Product Tabs, Smack Digital CDSEO Pro, AlteredCart Smart Search, AlteredCart One Page Checkout, Cart Works Power Filter, Firetank Software Feed Manager
Reply With Quote
  #2  
Old 01-12-2015, 12:09 AM
  qualiteam's Avatar 
qualiteam qualiteam is offline
 

X-Guru
  
Join Date: Dec 2010
Posts: 6,373
 

Default Re: XC5 Import Routine Slowing

Hello Mark,

Is it the stock X-Cart 5, or did you install any paid/custom modules?

Please could you list modules which are installed in your XC5 store?

Thanks!
__________________
Alex Solovev,
Qualiteam

---

User manual Video tutorials X-Cart FAQ

You are welcome to press "Thanks" button
if you find this post useful

Click here to learn how to apply patches

X-Cart Extensions
Reply With Quote
  #3  
Old 01-12-2015, 03:34 AM
  tony_sologubov's Avatar 
tony_sologubov tony_sologubov is offline
 

X-Cart team
  
Join Date: Jan 2009
Posts: 2,431
 

Default Re: XC5 Import Routine Slowing

Hi Mark!

Your CSV file looks OK, but the problem that each next 100 items are imported slowly that previous ones sounds new to me. Could you please send me the 5000-product CSV file, so I could try to recreate the problem on my local installation?

Tony.
__________________
Found a bug in X-Cart? Post it to our bug tracker!
Know how to make X-Cart better? Suggest an idea!
Reply With Quote
  #4  
Old 01-12-2015, 06:10 AM
 
Mark N Mark N is offline
 

Senior Member
  
Join Date: Sep 2011
Posts: 121
 

Default Re: XC5 Import Routine Slowing

To answer/respond:

- Yes, this is a stock installation, for testing purposes before posting with my issue I actually disabled all modules to try to isolate the problem, so the numbers I'm posting are run with no modules enabled whatsoever.

- Tony, I'll send you the file in a PM.

Graph of import times:

http://s18.postimg.org/wjdnf40dl/import_processing_times_1220_image001.png
__________________
X-Cart Gold Plus 4.6.5
Mods - WebsiteCM Dynamic Product Tabs, Smack Digital CDSEO Pro, AlteredCart Smart Search, AlteredCart One Page Checkout, Cart Works Power Filter, Firetank Software Feed Manager
Reply With Quote
  #5  
Old 01-15-2015, 06:32 AM
  tony_sologubov's Avatar 
tony_sologubov tony_sologubov is offline
 

X-Cart team
  
Join Date: Jan 2009
Posts: 2,431
 

Default Re: XC5 Import Routine Slowing

Hi Mark!

I tried to replicate the problem and I can see it on default installation. I have reported it to the team.
__________________
Found a bug in X-Cart? Post it to our bug tracker!
Know how to make X-Cart better? Suggest an idea!
Reply With Quote
  #6  
Old 03-07-2015, 11:44 AM
 
tonerguy tonerguy is offline
 

Advanced Member
  
Join Date: Feb 2015
Posts: 88
 

Default Re: XC5 Import Routine Slowing

In this recent version of Xcart 5.1.11 - 28,000 sku file are taking a rate of 8 hours to verify first 10,400 skus - this is way too long.

no extra modules installed - free version


my other application on same server can up load 40,000 skus in seconds.



Is there a fix to this - on your end.

Thanks
__________________
5.1.10
Reply With Quote
  #7  
Old 03-10-2015, 08:00 AM
 
xgarb xgarb is offline
 

eXpert
  
Join Date: Jul 2004
Location: UK
Posts: 262
 

Default Re: XC5 Import Routine Slowing

A desktop app that communicates directly via MySQL remote access would be great. Or even one that remotely logs into X-cart and works via PHP but all the grunt work done locally.

The problem with the import is that it's running in one thread due to the way PHP works. I guess it does something to stop timeouts or resource problems as well so it ends up being slowwww!
__________________
Core version: 5.3.xx
PHP: 5.6
MySQL: 5.5.50-cll
Web server: Apache
Reply With Quote
  #8  
Old 03-10-2015, 11:43 AM
  tony_sologubov's Avatar 
tony_sologubov tony_sologubov is offline
 

X-Cart team
  
Join Date: Jan 2009
Posts: 2,431
 

Default Re: XC5 Import Routine Slowing

Hi guys!

There is indeed a problem with import being slow. The problem is already solved in 5.2 version that will come up next, so please stay tuned for an update.

Tony.
__________________
Found a bug in X-Cart? Post it to our bug tracker!
Know how to make X-Cart better? Suggest an idea!
Reply With Quote
Reply
   X-Cart forums > X-Cart 5 > Dev Questions (X-Cart 5)


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -8. The time now is 06:47 AM.

   

 
X-Cart forums © 2001-2020