Linked elements not always created

[zebrafilm]

In both my projects I have trouble with the elements not being (fully) copied over when I create a new list pointing at the same database table.

On my 600+ element table I get max 50 in the new list, (J2.5 project)
in the second project (j3) I did not get the elements that were disabled (no not trashed) when I create the new list.

In the J2.5 project I have tried several times to make a new list but always get only a few items. Never the complete set.

In the J3 project I was able to create a new list and get all the elements in after I enabled them all.

1. Is this the default behavior? (disabled elements don't copy over to new lists)
2. Is there a way to manually import / create those missing (linked) elements?

 

[cheesegrits]

Hmmm. A 600 element table is pushing the envelope in a LOT of ways, and you could be hitting all kinds of limits we've never really anticipated. Our general rule of thumb is, if you have more than about 100 elements, then you probably need to rethink your database design. At 600, you are starting to run up against limitations within MySQL on things like max query length.

1. It may be the default behavior, but I don't think it should be. We should probably create them on the copy, and disable them. I'll raise a ticket on this.

2. Have you tried "update database" on the copy of the form? I don't offhand know if that'll fix it, and I suspect not, but it may be worth a go. Although to be honest, I'd recommend doing a database backup first.

If that doesn't do the trick, the only option would be for me to fix the underlying issue, then you zap the copy and do it again. That's specifically the J! 3.x one with the disabled element issue.

To be honest, the chance of us fixing the "600 elements on a J! 2.5 site" one is not very high. J! 2.5 is a dead product, and it's one of those corner case issues which would take a long time to set up a test case for, debug and fix (if its even fixable, I can think of several limitations which could be causing this which wouldn't even be fixable, to do with the J! 2.5 database API) which is very hard to justify when we're currently working hard to fix J! 3.2 related issues.

-- hugh

 

[zebrafilm]

Thanks for the feedback.
About my giant tables setup:

This was my first project and Fabrik seemed chaotic to say the least, now I have more overview and feel much more comfortable with the concept. Partly thanks to all the help here in the forum.
But I have to finish this first project for a paying customer so please lets find the best way.

Setup remains those 600 questions, each of them consisting of 2-3 radio buttons. My client put them in himself to save costs and I would not like to repeat this exercise.
When we first spoke, the issue of this oversized setup came up already and then it was suggested to at least split them up in different lists and point them to the same table. I have done exactly that (6 lists/forms) and that all works.
I can imagine that when people start filling in forms it indeed gets too heavy for MYSQL on a small server. (although running on SSD's)

J2.5 and migration: I am with you that it is good to work on the most recent and up to date system but 2.5 is not dead, it will be around April 2014. Anyway I am not against moving to J3.2 and Fabrik3.1.if that would help a lot.
Rob advised against it a few days ago, Hugh seems to favor this roadmap. I am also a fan of the latest F3.1 and indeed it is more efficient to work out issues in the latest version.
Moving the whole site to J3.2 will be a big job but for now I have no issue to run them side by side.

The main question will be, is it possible to split up the table into smaller ones, without redoing all the work done?

I can imagine that I copy the current table 5x , point the 6 lists each to the new tables and manually try to remove as many fields/columns as possible, comparing them to the elements of that list.
Would this be the best options or would it give trouble. Are there other ways besides this manual option?

Bastiaan

 

[zebrafilm]

Update:
OK created a new J3.2 website, added only the Fabrik elements and groups tables from the old site via phpmyadmin. Took all linked elements out to keep everything as clean as possible.
Then I recreate 6 lists and added the groups again to the right forms. Worked pretty good. Needed to help a bit in PHPmyadmin but in general it seemed ok.
Some elements got an acces ID of 5 (Guest) instead of 1 (Public) but with some manual editing everything seems good.
Now the only thing pretty big is the Fabrik elements table, all the list/form tables are now also split up.

I will need to keep the additional lists/forms a small as possible to I try to use only the current list for reporting and export. So far it works still speedy.
The calculations I planned to have inside the main list/form should now go to an external list again to minimise the overhead there. Still I am afraid my element list will grow close to 1000 elements....

What I missed in the whole process was some batch command in the J backend. Like adding a whole selection of elements to a group or set their access levels in one go.
Same when you have a lot of groups it feels a bit strange in the form editing that I can't select the whole bunch and add them in one go. Points for the wish-list.

Now I have to find a way to lock the inputs of one form into the next one. Of course I can let it jump to the right URL but they will need to get the same ID or something similar otherwise I will not be able to get the totals right for the six lists. (I am already using the userID but a user might add multiple records) Just using the ID might not use if someone starts to enter data in form 1 and does not continue through all of them.

Any thoughts about this appreciated!.

 

[cheesegrits]

I have some sites with upwards of 5,000 elements, so I wouldn't worry about how many elements in total. You can always filter the element display to just the form or group you want. The main issue is not having too many on one List.

So you are using redirect plugins to get from one form to the next?

If so, I'd just add something like an 'orig_rowid' or some such, and add it to the redirect URL with '&orig_rowid={rowid}' (on the first form), and have a hidden field element (or even join element) with an eval'ed default of:

$app = JFactory::getApplication();
return (int) ($app->input->get('orig_rowid', 0);

... or you could make the query string arg have the same name as the next step in the chain, like '&form2__orig_rowid={rowid}', and that setting of the default value should then happen automatically.

Then for each additional step in the chain, redirect using that orig_id element's value ... so form2 redirects to form3 with 'orig_rowid={form2___orig_rowid}'.

You should then end up with all forms in the chain having an orig_rowid which matches the rowid of the "prime" form. if those elements are joins, or you have copies of those lists/forms in which they are joins, then they should be easy to associate with the prime form.

-- hugh