Archived from groups: comp.sys.palmtops.pilot (More info?)

I would like to get the T5, but have been put off by the various bugs
and problems. Have those been worked out yet?

bblackmoor
2004-12-12

Archived from groups: comp.sys.palmtops.pilot (More info?)

On Mon, 13 Dec 2004 22:22:54 -0500, Brandon Blackmoor had this to say...


> I would like to get the T5, but have been put off by the various bugs
> and problems. Have those been worked out yet?
>
> bblackmoor
> 2004-12-12
>

What bugs?

--
Hope this helps.
Jim Anderson
( 8(|) To email me just pull my_finger

Archived from groups: comp.sys.palmtops.pilot (More info?)

"Jim Anderson" <fro2750@frontiernet.my_finger.net> wrote in message
news:MPG.1c28bfa077a7524a98994b@news.frontiernet.net...
> On Mon, 13 Dec 2004 22:22:54 -0500, Brandon Blackmoor had this to say...
>
>
>> I would like to get the T5, but have been put off by the various bugs
>> and problems. Have those been worked out yet?
>>
>> bblackmoor
>> 2004-12-12
>>
>
> What bugs?

Termites.

>
> --
> Hope this helps.
> Jim Anderson
> ( 8(|) To email me just pull my_finger

Archived from groups: comp.sys.palmtops.pilot (More info?)

Necron 99 wrote:
>>> I would like to get the T5, but have been put off by the
>>> various bugs and problems. Have those been worked out yet?
>>
>> What bugs?
>
> Termites.

(sigh)

Okay. I'll take that as a "no".

bblackmoor
2004-12-15

Archived from groups: comp.sys.palmtops.pilot (More info?)

On 2004-12-15, Brandon Blackmoor <bblackmoor@blackgate.net> wrote:

> (sigh)

Hehe, it's annoying when that happens ;-)

What bugs do you mean though, perhaps if you name the ones you know,
any users of the T5 present might be prompted to respond.

--
For every expert, there is an equal but opposite expert

Archived from groups: comp.sys.palmtops.pilot (More info?)

Ian Rawlings wrote:
> What bugs do you mean though, perhaps if you name the ones you know,
> any users of the T5 present might be prompted to respond.

These are the ones I know of:

1. PDB records take 512 bytes minimum, even if they only store 1 byte
of information, so your data might be 10 times as big as you
expect from how it was stored on other devices.

2. Applications that improperly used DmQueryRecord() where they
should've used DmGetRecord() will possibly cause databases to
get corrupted (whereas previously they only screwed up
hotsyncing), since the T5 uses this info to determine whether
to write data back to flash.

3. DB Cache is limited to 10 MB, so if you have a PRC bigger than
10 MB, it cannot ever be opened. And other similar limitations
exist when you hit the 10 MB barrier in various ways.

4. If two databases have names differing only in uppercase vs.
lowercase, this is supposed to be legal, but one overwrites
the other on the T5 because the T5 uses the FAT filesystem
(which is case insensitive) to store databases when it writes
them to flash.

5. I haven't verified this one fully yet, but it seems that if
you use sample code from PalmSource for handling screen
resizing, then it can cause the T5 to go into an infinite loop
in some cases when an alert (little warning or error window)
pops up.

There may be others, but those are the ones I'm aware of.

- Logan

Archived from groups: comp.sys.palmtops.pilot (More info?)

Ian Rawlings wrote:
>
> What bugs do you mean though, perhaps if
> you name the ones you know, any users of
> the T5 present might be prompted to respond.

I don't own a T5, so I can only tell you the ones I recall, but the two
that I recall are the slow memory (really, really slow memory, like
taking ten or fifteen seconds to start an app slow) and the calendar
crashing when you look at a particular view.

bblackmoor
2004-12-17

Archived from groups: comp.sys.palmtops.pilot (More info?)

<< Ian Rawlings wrote:
>
> What bugs do you mean though, perhaps if
> you name the ones you know, any users of
> the T5 present might be prompted to respond.

I don't own a T5, so I can only tell you the ones I recall, but the two
that I recall are the slow memory (really, really slow memory, like
taking ten or fifteen seconds to start an app slow) and the calendar
crashing when you look at a particular view.
>>

And the fact that a file occupies a MINIMUM of 512 KB, regardless of the files
actual size.

Dennis B. Swaney
remove .zz to reply

Archived from groups: comp.sys.palmtops.pilot (More info?)

ROMAD wrote:
>
> And the fact that a file occupies a MINIMUM
> of 512 KB, regardless of the files actual size.

I hadn't heard that one. That seems pretty serious on a device with
memory measured in MB.

bblackmoor
2004-12-17

Archived from groups: comp.sys.palmtops.pilot (More info?)

Brandon Blackmoor wrote:

> ROMAD wrote:
>
>>
>> And the fact that a file occupies a MINIMUM
>
> > of 512 KB, regardless of the files actual size.
>
> I hadn't heard that one. That seems pretty serious on a device with
> memory measured in MB.
>
> bblackmoor
> 2004-12-17

Look back up in this thread for Logan Shaw's posting. His first item is
a 512-byte (half-KiloByte) minimum "cluster" size (to use FAT fs terms)
for PDB records, which makes much more sense.

--
--
ROC

Archived from groups: comp.sys.palmtops.pilot (More info?)

On 2004-12-18, ROC <NoSpam@for.me> wrote:

> Look back up in this thread for Logan Shaw's posting. His first
> item is a 512-byte (half-KiloByte) minimum "cluster" size (to use
> FAT fs terms) for PDB records, which makes much more sense.

Now I'm confused. If a file takes a minimum of 512 bytes then that's
totally standard and isn't to be worried about, however if a pdb
RECORD takes a minimum of 512 bytes then that's more of a problem
because a single file could have thousands of PDB records in it (I
think?) each only being a few bytes long. And of course if an entire
file takes up a minimum of 512Kilobytes then that's not too good
either.

I think there's a lot of confusion on this one.

--
For every expert, there is an equal but opposite expert

Archived from groups: comp.sys.palmtops.pilot (More info?)

Ian Rawlings wrote:
> On 2004-12-18, ROC <NoSpam@for.me> wrote:
>>Look back up in this thread for Logan Shaw's posting. His first
>>item is a 512-byte (half-KiloByte) minimum "cluster" size (to use
>>FAT fs terms) for PDB records, which makes much more sense.

> Now I'm confused. If a file takes a minimum of 512 bytes then that's
> totally standard and isn't to be worried about, however if a pdb
> RECORD takes a minimum of 512 bytes then that's more of a problem
> because a single file could have thousands of PDB records in it (I
> think?) each only being a few bytes long.

I can see where that would be confusing.

First of all, you're right that you can have thousands of records
in a single PDB. For instance, every appointment in the datebook
is a separate record, all within one PDB.

The story I have heard on the T5 (and Treo 650) from Palm is this:
each *record* uses multiples of 512 bytes of flash for performance
reasons. Based on what I've inferred from their comments, this
doesn't really have anything much to do with cluster sizes directly.
Instead, it has to do with the type of flash they're using. It's
called NAND flash, and unlike some other memory, it doesn't allow
you to write only a few bytes at a time; instead, you must write a
whole 512-byte block even if all you want to change is a single byte.

So, they responded to this limitation by ensuring that no two PDB
records ever share one of these 512-byte blocks. If they did share,
then in order to modify a record (without unintentionally modifying
other records adjacent to it), you'd need to read the 512-byte block
into RAM, modify only part of it, and then write the 512-byte block
back out to flash. But if two PDB records never share a 512-byte
block, then when writing out a modified PDB record, you can simply
look at your PDB record's contents and create a new 512-byte block
and blast it out there without worrying about preserving anything
that was in that block before. So, very roughly speaking, the
method Palm chose is twice as fast as the obvious alternative.

Things are even more complicated and confusing when you realize
one other property of PDBs: you can change the size of a record
(or even delete a record) in the "middle" of the PDB. If you
have records 0, 1, 2, 3, and 4, you can change the size of
record 2 from 123 bytes to 6543 bytes without affecting the
others. This is not how regular files on regular computers work.
So, in order to make that happen in flash, it would probably
make things a bit easier if you restrict everything to 512-byte
boundaries. So that may be another part of the motivation.

If all that was too complicated, then here's the practical
implications:

(1) Palm's decision wasn't totally lame-brained; there is a
good technical reason to do it how they did.
(2) If you have a PDB with 10000 records in it, and they
each contain 10 bytes of data, the minimum theoretical
size it could be is 100,000 bytes. On regular Palms,
the per-record overhead is I think 14 bytes, so it would
be 240,000 bytes. On a Tungsten|T5 or Treo 650, it would
be at least 5,120,000 bytes. In other words, 20 times
as big.
(3) The 512-byte-per-record thing only has a negative effect
for PDBs that have a large number of small records.
If you had a PDB with 100 records of each 10 KB a piece,
the T5 and Treo 650 will perform no worse than any other
device.

Hope that helps.

- Logan

Archived from groups: comp.sys.palmtops.pilot (More info?)

Ian Rawlings wrote:
> increase during the product lifetime. NAND requires forward error
> correction (not sure on overhead, article states 1-4 bits of
> correction data but not whether this relates to each page which is
> unlikely or each byte which would be nasty)

1-4 bites per byte isn't really that bad at all. That's about the
normal level of redundancy you'd expect if you will be needing to
both detect and correct errors. It's just built into the process
of manufacturing it, so that when you want to make a chip with
1 megabit of useful storage, you shoot for 1.5 of actual bits
and call the end product "1 megabit with error correction".

> It sounds like they aren't loading pdbs into RAM before accesing them
> for speed purposes, as this would do away with the 512-byte minimum
> block limitation on records,

That is correct. According to the PalmOne documentation, they
only load the individual records as they are used. This makes
sense because Palm OS has two calls that let the application
tell the system that it wants to access a PDB record: DmGetRecord()
and DmQueryRecord(). One tells the system you'll be doing read-only
and the other tells it you want read-write access to the record.
Then the system writes back the data when you close the database,
when the RAM is running low and it needs to free some up by shuffling
some stuff out to flash, or when the power is turned off.

> with so much available space I don't
> think this is too bad an issue but it does sound inefficient if a
> single telephone number in a contacts database is stored as an
> individual record (taking 512 bytes) but not so bad if an entire
> contact including all addresses, telephone numbers etc is stored as a
> single record.

In the regular Palm OS address book, I believe each contact is
stored along with all its information in a single record. So
it's really not that bad for contacts. But for items on the
to-do list, it's really not that great since they might actually
be only 10 bytes. For example, I do my grocery list as individual
to-do items under a category called "Shopping", and just putting
"onions" or "soap" in there doesn't take a whole lot of space.
So a fair amount will be wasted with the 512-byte minimum.

> This is certainly the worst case scenario, but having tiny records is
> inefficient in any database due to the overheads associated with
> tracking record location and allocation so such implementations would
> be bad on any platform, just even worse on a NAND-based platform.

Yes, it is kinda bad in general, but on traditional Palms, the overhead
didn't really need to be that bad and wasn't that bad. On a traditional
Palm, everything is stored in battery-backed memory, so if you need
to store (say) 22 bytes, you can do so without much waste. That's sort
of part of the charm of the Palm device -- that there was an easy
(relatively), lightweight way to store small little amounts of data
without having to create a file for everything or define a special
file format and mess with reading the file and locating records
within it.

By the way, I can't say that the 512-byte "bug" (really a design
decision) will never be fixed. There are ways to address the
problems that led to this decision. One way is, every time you
read a 512-byte block, even if you only use 16 of the bytes,
keep the entire 512-byte block around in RAM *somewhere*. If
you need to change the bytes, read the 512-byte block from the
place, modifying it, and then write it out again. This avoids
the extra write that they were trying to avoid by doing everything
in 512-byte increments. And yes, it wastes RAM, but you can
limit the amount of RAM it wastes pretty easily: in the worst
case, you simply toss out these blocks that you are keeping
around and re-read from flash. That's exactly what you're
trying to avoid, but if you are smart about how you manage just
which blocks you keep and which you discard, it should still be
able to give you really good performance without wasting space
on the flash.

- Logan

Archived from groups: comp.sys.palmtops.pilot (More info?)

On 2004-12-19, Logan Shaw <lshaw-usenet@austin.rr.com> wrote:

> 1-4 bites per byte isn't really that bad at all. That's about the
> normal level of redundancy you'd expect if you will be needing to
> both detect and correct errors.

Yes, I'm not sure if they are actually correcting or just detecting,
and am also not sure if the error correction/detection is built into
the NAND chips or must be implemented by the developer. If they are
just detecting (so they can simply perform a re-read until it gets it
right) then you could get away with simple parity bits per byte
depending on the level of bit-flipping errors and the likelihood of
two bits being flipped in the same byte, or perhaps four bits for
every three bytes.

Of course the bit-flipping errors might be so infrequent that they
*can* get away with 1-4 bits per 512-byte block. Not enough data for
us to guess on this one ;-)

> By the way, I can't say that the 512-byte "bug" (really a design
> decision) will never be fixed. There are ways to address the
> problems that led to this decision. One way is, every time you read
> a 512-byte block, even if you only use 16 of the bytes, keep the
> entire 512-byte block around in RAM *somewhere*. If you need to
> change the bytes, read the 512-byte block from the place, modifying
> it, and then write it out again. This avoids the extra write that
> they were trying to avoid by doing everything in 512-byte
> increments. And yes, it wastes RAM, but you can limit the amount of
> RAM it wastes pretty easily: in the worst case, you simply toss out
> these blocks that you are keeping around and re-read from flash.
> That's exactly what you're trying to avoid, but if you are smart
> about how you manage just which blocks you keep and which you
> discard, it should still be able to give you really good performance
> without wasting space on the flash.

I'm not sure what you were trying to say above came across very well,
but I suspect you mean something like the way that modern operating
systems do disc cacheing on writes, perhaps flushing the write cache
when the device power is turned off or once every 10 seconds or so.
This brings in a host of other problems though if you start splitting
multiple records over 512-byte boundaries as enlarging a record can
have a snowball effect so perhaps they're best off leaving it as it
is, respecially as they're apparently moving to a linux kernel-based
system. I'm not sure how much of the kernel they will use but there
is already work being done on NAND-flash filing systems by the linux
developers (JJFS I think) so perhaps Palm just slapped this together
as a temporary solution ;-)

--
For every expert, there is an equal but opposite expert

Archived from groups: comp.sys.palmtops.pilot (More info?)

On Sun, 19 Dec 2004 08:30:16 +0000, Ian Rawlings wrote:

> > 1-4 bites per byte isn't really that bad at all. That's about the
> > normal level of redundancy you'd expect if you will be needing to
> > both detect and correct errors.
>
> Yes, I'm not sure if they are actually correcting or just detecting,
> and am also not sure if the error correction/detection is built into
> the NAND chips or must be implemented by the developer.

One bit per byte would be enough to detect errors. As I recall
back in the days of the 486 and Pentium computers, non-parity SIMMs
were 32-bit wide memory, and parity SIMMs had 36 bits, ie, 1 extra
bit per byte. I don't recall how wide the ECC SIMMs were, but I'm
pretty sure that they didn't need anything like 4 bits/byte. I think
2 bits/byte would be sufficient, which is still a lot of memory (an
extra 25%). By 'developer' I assume that you mean the designer of
the hardware? It certainly wouldn't be implemented by application
developers, which is what usually comes to mind when someone says
'developer'.

Archived from groups: comp.sys.palmtops.pilot (More info?)

On 2004-12-19, BillB <rainbose@earthlink.newt> wrote:

> One bit per byte would be enough to detect errors.

It depends on the error rate, it's possible for two bitflips in a byte
to preserve the same parity so the parity bit would still match the
byte despite there being two bits having been flipped.

At any rate we don't know enough about it to really speculate, with
the right information (error rates, acceptable losses). The error
rate with these things is much higher than with the ECC SIMMs you talk
about later in your post.

> By 'developer' I assume that you mean the designer of the hardware?
> It certainly wouldn't be implemented by application developers,

It would either be implemented by the hardware developer or the
*operating system* developer. Error correction/detection is not
always needed, e.g. for the use of storing audio or video data so the
developer would not necessarily want to eat up valuable space with
needless error correction. The hardware manufacturer can leave it up
to the purchaser to decide how to best do it.

As an example, an audio CD can store more audio data than a data CD
because there is no error correction as the human ear doesn't notice
the occasional errors, a data CD loses a bit less than 100 megs of
useable space through the forward error correction used to compensate
for the bit error rates of the CD media. The loss of space is only
partially down to the formatting of the filesystem, contrary to what
some people believe.

So if you think that error-prone storage is a strange concept, you've
been using it for years ;-)

--
For every expert, there is an equal but opposite expert

Archived from groups: comp.sys.palmtops.pilot (More info?)

Ian Rawlings wrote:

> On 2004-12-19, BillB <rainbose@earthlink.newt> wrote:
>
>> One bit per byte would be enough to detect errors.
>
> It depends on the error rate, it's possible for two bitflips in a byte
> to preserve the same parity so the parity bit would still match the
> byte despite there being two bits having been flipped.

A parity bit will detect single-bit errors, but it can't correct them, so in
something as error-prone as being described, you'd want more than that.

> At any rate we don't know enough about it to really speculate, with
> the right information (error rates, acceptable losses). The error
> rate with these things is much higher than with the ECC SIMMs you talk
> about later in your post.

One of the previous posters mentioned having 1.5x as much storage for error
detection and recovery. A Hamming Code takes 3 bits per byte, and can
detect and recover from 1 bit errors in a byte, and detect (and maybe
recover from -- I can't remember off-hand) 2 bit errors in a byte as well.

3 extra bits per byte would be roughly 1.5x the memory, so that might be
what's being used.

--
ZZzz |\ _,,,---,,_ Travis S. Casey <efindel@earthlink.net>
/,`.-'`' -. ;-;;,_ No one agrees with me. Not even me.
|,4- ) )-,_..;\ ( `'-'
'---''(_/--' `-'\_)

Archived from groups: comp.sys.palmtops.pilot (More info?)

On Sun, 19 Dec 2004 13:25:33 +0000, Ian Rawlings wrote: