import timeout« Back to Previous Page

We are having issues with importing the feed. The max timeout of the server is 120 seconds and the host wont change it. We are using the Street CRM integration. There are just under 110 properties. What is strange is when I was researching the other issues that I've reported, I put logging on the feed. It quickly loads about 4.5mb of data in, but by the time the import has finished it had logged the data out tons of times. I'm just looking at it now, and the debug log ended up with 27 copies of the data output. Which doesn't equal the number of properties. Can you offer any feedback on this? Do you expect the feed to take over 120 seconds with around 110 properties? We keep ending up with partial syncs. If this is expected behaviour, I'm wondering if the import process could be done differently, with batches being imported via rest calls, so that the entire batch doesn't have to be done in one request.
Posted by Ben Sykes
Asked on 13th July 2023 9:16 pm
▲
0
▼

Sorry for not getting back to you before. Just to explain how the Street import works we query their API to retrieve all the properties they list on their system but this in done in a Paged fashion, so we obtain 10 properties at a time then request the next page of properties (another 10) and so on until we get to the end of the listed properties. So, I would expect you to see a number of requests to the API endpoint. We also request for each property all the rooms, image, floorplans and more so there is a lot of data for each property that is collected. Obviously, the requests all take time to request and deliver and WordPress has to process these and store all the data which takes time. It doesn’t help that the WordPress DB structure is not at all optimised for storing property data and that makes processing slow. As a result it really helps if we can have extended timeouts for processing of feed data. I have issued a plugin update that increases the page size for API request to 100 (the maximum we can request) in order to cut down the number of API requests which may help.

Posted by Ian
Answered on 19th July 2023 3:25 pm
▲
0
▼

Can you acknowledge that you have seen this and give me any kind of update please? The client is getting pressured to resolve this ASAP as the developers of the new builds which they are supposed to be advertising are very unhappy that their listings are no longer appearing on the website.

Posted by Ben Sykes
Answered on 18th July 2023 11:10 pm