Hi again,

You learn as you work.
1. Most (but not all) of my problems below were related to using a non-UTF8 file.
2. I am now fairly convinced that UTF8 works well most of the time.

BUT, now that I think that I do everything right, I am still at - "firstFieldName" not valid... Any ideas?


---original post below---
I downloaded and installed current Joomla, CB and CBJuice. (Versions 3.4.1, 3.3.2, and 3.3.2 (beta) respectively, I wanted to upload a group of people as users. None of them has a valid (known) email adress. We are Scandinavians, så many names include the scandinavian characters å, ä and ö (and their capital counterparts). Here is what I experienced so far.

First I downloaded the full set of names to notepad, copied it to Word to be able to replace the commas with TABs and pasted the file into Excel.
-- I noted that the "params" field in itself contains comma separators, which resulted in too many TABS. However easily fixed. (Either by exporting semicolon separated or by fixing in Excel).

To create the import csv I first copied from Excel to notepad (to get rid of Excel special formatting), then from notepad to Word to replace the TABs with commas, then back to Notepad to save as UTF-8 .csv
-- The first field name was always rejected (regardless of which it was). The report said "name" not valid (yes, with """ around the name, although the remaining field names appeared stripped and clean.
-- I later tried to open that same file into Libre Office and save it again from there. The file names were all good then. Obviously (although I have "show invisible characters" turned on and could not see any suspicious characters anywhere in the file), Notepad _does_ create the first field name (at least) to be improperly decoded. (Edit: Or not, see below)

From then on I used Libre Office as the medium of the final save.

However, the swedish characters played havoc with the data and the import was rejected for various reasons. I will investigate that further (maybe LibreOffice did not save in UTF-8).
(off to do some more testing
It turned out it did not, so I changed the saving from LibreOffice to UTF-8 (and saved with <CR><LF> as line terminator), and the very same problem appeared again, namely...

"name" is not valid
Terminating because of undefined header variables

...which leads me to believe that there has never been a problem with the field names. BUT that when UTF-8 is specified AND Scandinavian characters are in the data, THEN this is how the inability to cope with names including scandinavian characters manifests itself.

I have succeded in some tests but then the LastName was truncated just before the scandinavian character. (and when CBJuice tried to make an (invalid) email adress from the garbled up name it got even worse).

It does not seem that CBJuice can handle Scandinavian characters, not even when UTF-8 is specified. My thought is that CBJuice ought to be UTF-8 compliant when that is the preferred import means.

I include two of my testfiles for verification

Libeoffice is easy. check the "edit filter settings" to get to set the character set. The easiest way to convert from and to UTF8 is to use notepad which has a character set option. Notepad++ has more sophisticated options but is not needed for that simple use.

Yes, I found the UTF-8 setting in LO. However, see my edits above. (CBJuice2 does not handle scandinavian UTF-8 characters properly)