Int/float converted to Decimal by collective.exportimport

zopyx · April 12, 2025, 9:30am

I am currently running a migration to Plone 6.1.1 using latest collective.exportimport version.

We have a content-type with fields defined as Float or Int. They are correctly exported as floats or ints on the original Plone 4 sites. They are recreated a Decimal types in Plone 6.1. The problem is here is the usage of ijson which converts intentionally to Decimal.

github.com/isagalaev/ijson

Numeric decimals being converted into Decimal() objects

opened 05:19PM - 07 Sep 13 UTC

closed 05:28AM - 10 Sep 13 UTC

mittenchops

I'm reading from a geographic shapefile that has, among other data types, a list… of coordinates, such as... ``` { "type": "FeatureCollection", "features": [ { "geometry": { "type": "Polygon", "coordinates": [ [ [ -123.09098499999993, 45.77992400000005 ], ``` The coordinates are getting converted in ijson into: ``` [Decimal('-123.09098499999993), Decimal('45.77992400000005')] ``` I'm not sure whether this is a bug or a feature, but it's making my database choke. =) My ijson loader looks like this: ``` def loader(shapefile, collection): jf = open(shapefile) data = ijson.items(jf,'features') for d in data: try: db[collection].insert(d) except: pass ``` My vanilla loader, without ijson, looked like this (and worked, but not on the large files I need ijson for) ``` def loader(shapefile, collection): jf = open(shapefile) data = json.load(jf) for d in data['features']: db[collection].insert(d) ``` My database is complaining about the Decimal() objects. Where would I add something to convert only the Decimal() objects into their numeric values?

However, this behaviour is not acceptable for a Plone migration. A float in the old system must be a float in the new system. Is there a straightforward solution, or should I patch collective export-import?

mtrebron · April 12, 2025, 11:27am

Thanks for linking to an entertaining read. Great feature/not bug: "No, a round peg has to be made square or else it won't fit in some boards"

zopyx · April 12, 2025, 12:15pm

My workaround for this particular migration is here: