Here's a simple diagram I drew up to summarize what I have come to understand over the past few months of research: Much more thought, in terms of the intangible properties - aesthetic, intention and perception, should be put into selecting the right materials, that must touch the body than materials that will be far away.
For example, the Jawbone Up! puts rubber in contact with the user's skin 24x7, it's uncomfortable and gets caught up in clothing. It's matte look is appealing, the rubber's intention may be to hold onto the arm/wrist in order to provide better movement measurement. Perception? The matte rubber (especially in black) in contrast with plastic painted with a metallic finish gives the device a jewelry-like appearance, therefore to the casual observer it's just another piece of decoration and not immediately a device that helps one keep track of physical activity.
Another example would be Walter De Maria's Broken Kilometer installation, where the brass rod in the back of the room, far away from the viewer, could be made of any material made to look like bronze.