qx::clone failure

Open discussion on QxOrm library

qx::clone failure

Postby Kasuax » Fri Nov 04, 2016 5:20 am

I'm experiencing a new issue with the qx framework. I am unable to use the qx::clone functionality as advertised with my complex tree structures.

If I clone my structure and motion to delete it, I receive QList<Type> double delete issues. I confirmed by enabling QT_SHAREDPOINTER_TRACK_POINTERS. Somehow the boost framework (I think) is rebinding the same QSharedPointer<T> into multiple QList. What I can say is that my original structure would have exhibited the same QSharedPointer<T> in multiple places throughout my tree. That said, I was under the impression that boost could follow and reuse identical pointers across the structure.

On a possible related note. What does _QX_ENABLE_BOOST_SERIALIZATION_POLYMORPHIC do?
Many of my objects have abstract base classes.

At any rate I know I'm not giving you a lot to work with. I'm hoping to spur some conversation. Tomorrow I could provide a stack trace and more.

Thanks,
SH
Kasuax
 
Posts: 62
Joined: Mon Jun 20, 2016 6:42 pm

Re: qx::clone failure

Postby qxorm » Fri Nov 04, 2016 8:46 am

Hello,

I had recently a similar issue with qx::clone() function developing QxEntityEditor (which is based on QxOrm library).
The problem was also a double deletion of the root item of a tree, which is also referenced somewhere else in the tree.
The problem was not in qx::clone(), but was the way I called it :

I first wrote something like this (I'm using boost::shared_ptr<T>, but it is the same thing with QSharedPointer<T>) :
Code: Select all
boost::shared_ptr<myClass> clone;
clone.reset(qx::clone_to_nude_ptr(* other));


But this previous code can cause a double deletion of the root item of the tree if it is referenced also somewhere else in the tree.
To avoid that, I changed the code like this :
Code: Select all
boost::shared_ptr<myClass> * pCloneTemp = qx::clone_to_nude_ptr(other);
boost::shared_ptr<myClass> pClone = (pCloneTemp ? (* pCloneTemp) : boost::shared_ptr<myClass>());
if (pCloneTemp) { delete pCloneTemp; pCloneTemp = NULL; }
// Now use pClone....


Now, I don't have any problem with double deletion.
So I would say :
- be confident about qx::clone() implementation (especially be confident about boost serialization if you enabled it) ;
- be careful about how you call it with smart pointers.

Note : another way to clone your instance and avoid a double deletion is to use the JSON serialization (but it is a slower solution), something like this :
Code: Select all
QString json = qx::serialization::json::to_string(myInstance);
boost::shared_ptr<myClass> clone;
qx::serialization::json::from_string(clone, json);
qxorm
Site Admin
 
Posts: 481
Joined: Mon Apr 12, 2010 7:45 am

Re: qx::clone failure

Postby Kasuax » Fri Nov 04, 2016 1:54 pm

Kasuax wrote:What does _QX_ENABLE_BOOST_SERIALIZATION_POLYMORPHIC do?
Kasuax
 
Posts: 62
Joined: Mon Jun 20, 2016 6:42 pm

Re: qx::clone failure

Postby Kasuax » Fri Nov 04, 2016 1:56 pm

qxorm wrote:Hello,

I had recently a similar issue with qx::clone() function developing QxEntityEditor (which is based on QxOrm library).
The problem was also a double deletion of the root item of a tree, which is also referenced somewhere else in the tree.
The problem was not in qx::clone(), but was the way I called it :

I first wrote something like this (I'm using boost::shared_ptr<T>, but it is the same thing with QSharedPointer<T>) :
Code: Select all
boost::shared_ptr<myClass> clone;
clone.reset(qx::clone_to_nude_ptr(* other));


But this previous code can cause a double deletion of the root item of the tree if it is referenced also somewhere else in the tree.
To avoid that, I changed the code like this :
Code: Select all
boost::shared_ptr<myClass> * pCloneTemp = qx::clone_to_nude_ptr(other);
boost::shared_ptr<myClass> pClone = (pCloneTemp ? (* pCloneTemp) : boost::shared_ptr<myClass>());
if (pCloneTemp) { delete pCloneTemp; pCloneTemp = NULL; }
// Now use pClone....


Now, I don't have any problem with double deletion.
So I would say :
- be confident about qx::clone() implementation (especially be confident about boost serialization if you enabled it) ;
- be careful about how you call it with smart pointers.

Note : another way to clone your instance and avoid a double deletion is to use the JSON serialization (but it is a slower solution), something like this :
Code: Select all
QString json = qx::serialization::json::to_string(myInstance);
boost::shared_ptr<myClass> clone;
qx::serialization::json::from_string(clone, json);


From first glance I don't see how you avoided the issue of double delete...?
Kasuax
 
Posts: 62
Joined: Mon Jun 20, 2016 6:42 pm

Re: qx::clone failure

Postby Kasuax » Fri Nov 04, 2016 1:59 pm

qxorm wrote:Note : another way to clone your instance and avoid a double deletion is to use the JSON serialization (but it is a slower solution), something like this :
Code: Select all
QString json = qx::serialization::json::to_string(myInstance);
boost::shared_ptr<myClass> clone;
qx::serialization::json::from_string(clone, json);


Instead of json, what if I just used the QByteArray serialize?

Also, it says somewhere in the documentation that without boost support you get limited binary serialization. Fair, but what does limited mean? If I have a custom type and use the Qt binary serialize do I need to define those transforms for my custom property types? I think somewhere in Qt documentation it specifies you can write QByteArray operator<<(customtype) stream operators. I've just never actually done it.

Thanks,
SH
Kasuax
 
Posts: 62
Joined: Mon Jun 20, 2016 6:42 pm

Re: qx::clone failure

Postby qxorm » Fri Nov 04, 2016 3:57 pm

What does _QX_ENABLE_BOOST_SERIALIZATION_POLYMORPHIC do?

It is explained in the manual here : http://www.qxorm.com/qxorm_en/manual.html#manual_630
This flag just enables another boost serialization module (the polymorphic one, more details on boost serialization documentation).
But I don't think it will help you with your issue.

From first glance I don't see how you avoided the issue of double delete...?

Just changing the way I called qx::clone() function, now it works with :
Code: Select all
boost::shared_ptr<myClass> * pCloneTemp = qx::clone_to_nude_ptr(other);
boost::shared_ptr<myClass> pClone = (pCloneTemp ? (* pCloneTemp) : boost::shared_ptr<myClass>());
if (pCloneTemp) { delete pCloneTemp; pCloneTemp = NULL; }
// Now use pClone....

As I said, the problem is the double deletion of the root item of your tree when it is referenced somewhere else in the tree.
boost serialization keeps a trace of a same pointer to reuse it later.
So if your root item is encapsulated in a smart-pointer somewhere else in your tree, and if you encapsulate also this root item directly in a smart-pointer yourself calling qx::clone(), then you have 2 smart-pointers on the same instance ==> double deletion ==> crash.

The 3 lines of code I gave you to call qx::clone() function (in fact qx::clone_to_nude_ptr() function) is just a way to protect the root item of your tree to double deletion (no 2 different smart-pointer which take ownership of a same raw pointer).

Instead of json, what if I just used the QByteArray serialize?

I think you mean Qt QDataStream serialization (using qx::serialization::qt namespace) : no, I tried it when I had the same issue with QxEntityEditor : the problem is when you enable boost serialization, QDataStream will continue to use boost serialization for some containers ==> so you will certainly have the same issue.
JSON serialization is safe because it will not use boost serialization at all, and it will not keep a trace of a same pointer used several times in the tree.

it says somewhere in the documentation that without boost support you get limited binary serialization. Fair, but what does limited mean?

I improved that in QxOrm 1.4.2 : the goal is to not have to deal with boost serialization, and this is why boost serialization is not required now by QxOrm library by default (it reduces also a lot compilation times).
So "limited" is less true now, but boost serialization provides many features like keep a trace of a same pointer for example.

If I have a custom type and use the Qt binary serialize do I need to define those transforms for my custom property types? I think somewhere in Qt documentation it specifies you can write QByteArray operator<<(customtype) stream operators. I've just never actually done it.

QxOrm library provides already a lot of operator<< to serialize automatically basic types, containers, smart-pointers, etc...
This is why you didn't have to write them until now.
All classes registered in QxOrm context can be serialized without writing these operators.
For custom types which are not supported by QxOrm library, it is explained in the manual here : http://www.qxorm.com/qxorm_en/manual.html#manual_460
qxorm
Site Admin
 
Posts: 481
Joined: Mon Apr 12, 2010 7:45 am

Re: qx::clone failure

Postby Kasuax » Tue Nov 08, 2016 5:34 am

No dice... My problem wasn't with the root item. I employed your fix and the same issue occurred. The items within my tree are being reused intentionally and I think that's causing the problem somehow.


Code: Select all
1   QBasicAtomicInt::deref                                                                         qatomic_windows.h          324 0x663c862d
2   QList<QSharedPointer<QXC::ComponentProperty>>::~QList<QSharedPointer<QXC::ComponentProperty>>  qlist.h                    731 0xf7eff72 
3   QXC::PropertySchema::~PropertySchema                                                           QXC_PropertySchema.gen.cpp 87  0xfe439d7 
4   QXC::PropertySchema::`vector deleting destructor'                                              QXC                            0xf7d615d 
5   QtSharedPointer::ExternalRefCount<QXC::PropertySchema>::deref                                  qsharedpointer_impl.h      342 0xf7ec408 
6   QtSharedPointer::ExternalRefCount<QXC::PropertySchema>::deref                                  qsharedpointer_impl.h      336 0xf7e9bd9 
7   QtSharedPointer::ExternalRefCount<QXC::PropertySchema>::~ExternalRefCount<QXC::PropertySchema> qsharedpointer_impl.h      401 0xf7e552f 
8   QSharedPointer<QXC::PropertySchema>::~QSharedPointer<QXC::PropertySchema>                      QXC                            0xf7d61af 
9   QSharedPointer<QXC::PropertySchema>::`scalar deleting destructor'                              QXC                            0xf84093f 
10  QList<QSharedPointer<QXC::PropertySchema>>::node_destruct                                      qlist.h                    431 0xf8408d1 
11  QList<QSharedPointer<QXC::PropertySchema>>::free                                               qlist.h                    759 0xf80d3bb 
12  QList<QSharedPointer<QXC::PropertySchema>>::~QList<QSharedPointer<QXC::PropertySchema>>        qlist.h                    733 0xf7efb47 
13  QXC::ChannelSpec::~ChannelSpec                                                                 QXC_ChannelSpec.gen.cpp    55  0xf9f6fea 
14  QXC::ChannelSpec::`vector deleting destructor'                                                 QXC                            0xf7d55dd 
15  QtSharedPointer::ExternalRefCount<QXC::ChannelSpec>::deref                                     qsharedpointer_impl.h      342 0xf7ec2f8 
16  QtSharedPointer::ExternalRefCount<QXC::ChannelSpec>::deref                                     qsharedpointer_impl.h      336 0xf7e9b59 
17  QtSharedPointer::ExternalRefCount<QXC::ChannelSpec>::~ExternalRefCount<QXC::ChannelSpec>       qsharedpointer_impl.h      401 0xf7e52cf 
18  QSharedPointer<QXC::ChannelSpec>::~QSharedPointer<QXC::ChannelSpec>                            QXC                            0xf7d562f 
19  QSharedPointer<QXC::ChannelSpec>::`scalar deleting destructor'                                 QXC                            0xf8405bf 
20  QList<QSharedPointer<QXC::ChannelSpec>>::node_destruct                                         qlist.h                    431 0xf840551 
... <More>                                                                                                                                   
Kasuax
 
Posts: 62
Joined: Mon Jun 20, 2016 6:42 pm

Re: qx::clone failure

Postby qxorm » Tue Nov 08, 2016 9:29 am

The items within my tree are being reused intentionally and I think that's causing the problem somehow.

Maybe, but this is strange because my tree also reuses items within the tree and I don't have any problem, the difference is that I use boost::shared_ptr<T> instead of QSharePointer<T>, but it should be the same thing.

Have you tried to convert your tree to JSON, then create a new tree instance based on the JSON stream ?
Something like this :
Code: Select all
QString json = qx::serialization::json::to_string(myInstance);
boost::shared_ptr<myClass> clone;
qx::serialization::json::from_string(clone, json);
qxorm
Site Admin
 
Posts: 481
Joined: Mon Apr 12, 2010 7:45 am

Re: qx::clone failure

Postby Kasuax » Tue Nov 08, 2016 3:44 pm

I can't use json because i'm not using Qt5... We're stuck on 4.8.6 for the time being.
Any other thoughts? None of the serialize mechanisms seem to be working.

Code: Select all
template <class T>
QSharedPointer<T> safeClone(const QSharedPointer<T>& item)
{
    QSharedPointer<T> clone;

    QByteArray array = qx::serialization::qt::to_byte_array(item);
    qx::serialization::qt::from_byte_array(clone, array);

    return clone;   
}


I guess I could try to switch over to boost pointers just to see if this is resolvable. Can you think of anything else we can try?
Kasuax
 
Posts: 62
Joined: Mon Jun 20, 2016 6:42 pm

Re: qx::clone failure

Postby qxorm » Tue Nov 08, 2016 4:30 pm

Any other thoughts ?

If you don't use boost::serialization in your application, then you could try to not enable it : do not define _QX_ENABLE_BOOST_SERIALIZATION compilation option.
Then, you will have the QDataStream serialization process which will work without using boost (qx::clone() function will be still available but will not use boost serialization).
qxorm
Site Admin
 
Posts: 481
Joined: Mon Apr 12, 2010 7:45 am

Next

Return to QxOrm - Open discussion

Who is online

Users browsing this forum: No registered users and 2 guests