Programmatically creating files and populating their fields

corrected typo in example code.

found the physical files! All other facts remain valid.

Thanks both for your suggestions. I was really after being able to do it using entities though, as much to understand how they work as anything else. I have yet to import nodes and may give the migrate module another look for that but I have managed to get the files import doing basically what I want with the following code:

// this code snippet is for a single entity.  Wrap it in a loop for multiple.
    // first check that the file exists
    if (file_exists($file))
    {
        // create a file object for Drupal
        $dfile = (object) array(
			'uri' => $file,
			'filemime' => file_get_mimetype($file),
			'status' => 1,
		);
		        
        // now get Drupal to copy it
        $drupalfile = file_copy($dfile, 'public://' . $folder_name . '/');
        
        // we should have a file object with an id.  The id is all we need
        $fid = $drupalfile->fid;
        
        // load the entity for this file (note, returns an array, indexed on fid
        $mediafiles = entity_load('file', array($fid));
        
        // set some attributes for the file and save those too
        // set the folder which is linked to a taxonomy term
        // check for a taxonomy term to get the ID
        $terms = taxonomy_get_term_by_name($folder_name);
        
        // we should have an array.  Find the value which is used by folders
        if (is_array($terms))
        {
        	foreach ($terms as $term)
        	{
        		if ($term->vocabulary_machine_name == 'media_folders')
        		{
        			$folderid = $term->tid;
                                break;
        		}
        	}
        }
                
        // set the file entity's folder
        if ($folderid) $mediafiles[$fid]->field_folder[LANGUAGE_NONE][0]['tid'] = $folderid;
       
        // change the filename so it uses something more meaningful
        $mediafiles[$fid]->filename = $file_name;
        
        // finally, save the attributes
        entity_save('file', $mediafiles[$fid]);

tsvenson, I did know that but categorising the content is crucial for this work. The media browser plus module adds folders and allows you to import into folders but at the time I was doing this it didn't actually work properly!

Thanks all.