Problem/Motivation

The method \Drupal\Component\FileSystem::getOsTemporaryDirectory() is trying to maintain multi-os compatibility instead of using php defaults.

Proposed resolution

Deprecate the method and replace it with sys_get_temp_dir().

Remaining tasks

None

User interface changes

None

API changes

The method \Drupal\Component\FileSystem::getOsTemporaryDirectory() is deprecated and replaced by
sys_get_temp_dir().

Data model changes

None

Release notes snippet

TBD

Original Issue

In #3039026-67: Deprecate file_directory_temp() and move to FileSystem service which adds "fallback" to configurable temporary dir the last fallback to this component which, according to https://www.php.net/manual/en/wrappers.php.php#wrappers.php.memory could be replaced with default stream wrapper
The whole component trying to maintain multi-os compatibility instead of using php defaults
The logic split into component & service could be simplified

Issue fork drupal-3063704

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

andypost created an issue. See original summary.

berdir’s picture

> The whole component trying to maintain multi-os compatibility instead of using php defaults

That depends on whether the php defaults are actually sufficient and we'll need to test that.

All those low-level functions we have in the FileSystem class are to work around and expand on incomplete/missing features in the corresponding php functions.

andypost’s picture

@Berdir basically this function trying to guess where temp dir lives but starts with upload_tmp_dir instead of sys_get_temp_dir() and environment variables

Meantime php itself going different https://github.com/php/php-src/blob/master/main/php_open_temporary_file....

I started to review places where core require temp dir and no longer see the reason why upload_tmp_dir used here.
When most of temp file/dir usage are in tests and upload is delegated to \Symfony\Component\HttpFoundation\FileBag.
Now it looks core need to reconsider temporary file/dir usage

zviryatko’s picture

@Berdir I've done a little research and found that this code was added 13 years ago (!) because there was an error on Windows 2000 and PHP 5.0.5-dev. Initial issue #26249 and it wasn't changed but only moved around.
Right now I think that there's no reason to keep it there. Seems we need to start getting rid of legacy in the core.

catch’s picture

catch’s picture

Would definitely be great to remove this if possible. I guess the first step is a patch and then try to get it tested on windows?

andypost’s picture

Assigned: Unassigned » andypost

Good point, will do stub

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

longwave’s picture

Discovered this issue while helping @mglaman and @mixologic figure out why the wrong tmp dir was being used in upgrade_status even though sys_temp_dir was being explicitly set.

$ php -d sys_temp_dir=/var/tmp vendor/bin/drush ev 'print \Drupal\Component\FileSystem\FileSystem::getOsTemporaryDirectory();'
/tmp

This is because \Drupal\Component\FileSystem\FileSystem::getOsTemporaryDirectory() looks at these directories in order of preference:

  • upload_tmp_dir
  • /tmp (or Windows equivalent)
  • sys_temp_dir

This feels like a bug to me, because if sys_temp_dir is set in php.ini I would expect any application to follow that - but if /tmp exists (and it almost always will on a Unix system) then it will always be used in preference.

andypost’s picture

Assigned: andypost » Unassigned

It looks a blocker now, but I'm afk

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

kim.pepper’s picture

Status: Active » Needs review
StatusFileSize
new6.98 KB

Initial patch that just swaps usage with 'php://temp'. Let's see what breaks!

andypost’s picture

Looks tests won't run

kim.pepper’s picture

Title: Deprecate \Drupal\Component\FileSystem\FileSystem::getOsTemporaryDirectory() in favour of php://temp » Deprecate \Drupal\Component\FileSystem\FileSystem::getOsTemporaryDirectory() and replace with sys_get_temp_dir()
StatusFileSize
new6.88 KB

From what I've read, 'php://temp' is not a location, but will create a new one-time-use temp dir each time the stream is opened, and cannot be re-used.

Instead we want to use sys_get_temp_dir().

Here's a patch that does that.

kim.pepper’s picture

StatusFileSize
new9.71 KB
new5.85 KB

Fix coding standards and adds a deprecation test.

berdir’s picture

+++ b/core/lib/Drupal/Component/FileSystem/FileSystem.php
@@ -13,34 +13,15 @@ class FileSystem {
    */
   public static function getOsTemporaryDirectory() {
-    $directories = [];
-
-    // Has PHP been set with an upload_tmp_dir?
-    if (ini_get('upload_tmp_dir')) {
-      $directories[] = ini_get('upload_tmp_dir');
-    }
-
-    // Operating system specific dirs.
-    if (substr(PHP_OS, 0, 3) == 'WIN') {
-      $directories[] = 'c:\\windows\\temp';
-      $directories[] = 'c:\\winnt\\temp';
-    }
-    else {
-      $directories[] = '/tmp';
-    }
-    // PHP may be able to find an alternative tmp directory.
-    $directories[] = sys_get_temp_dir();
-
-    foreach ($directories as $directory) {
-      if (is_dir($directory) && is_writable($directory)) {
-        // Both sys_get_temp_dir() and ini_get('upload_tmp_dir') can return paths
-        // with a trailing directory separator.
-        return rtrim($directory, DIRECTORY_SEPARATOR);
-      }
-    }
-    return FALSE;
+    @trigger_error(__METHOD__ . ' is deprecated in drupal:9.3.0 and is removed from drupal:10.0.0. Use sys_get_temp_dir() instead. See https://www.drupal.org/node/3225275', E_USER_DEPRECATED);
+    return sys_get_temp_dir();
   }
 

I think I'm pretty confused now.

Are we certain that we no longer need all those existing checks?

The trailing / inconsistency for example, and according to some comments on https://www.php.net/manual/en/function.sys-get-temp-dir.php, it might return a per-user path in some cases.

Also, https://www.php.net/manual/en/ini.core.php#ini.upload-tmp-dir does mention open_basedir, what about hosting setups that use that (is that still a thing these days?) and have a customized upload dir.

Feels like that might be a lot of work to verify all those things on all kinds of operationg systems to deprecate a handful lines of code?

kim.pepper’s picture

I guess we are testing those assumptions? Some of the comments are quite old. For example, trailing slash seems to always be removed https://github.com/php/php-src/blob/af6c11c5f060870d052a2b765dc634d9e47d...

sys_temp_dir can be set in php.ini since PHP 5.5 and we check an empty value in \Drupal\Core\File\FileSystem::getTempDirectory().

Feels like that might be a lot of work to verify all those things on all kinds of operationg systems to deprecate a handful lines of code?

You might be right. 🤔

andypost’s picture

Great point about temporary file (within request) and it's make me to check current usage of sys_get_temp_dir() in core.
The file system service just allows to decorate PHP and then OS-settings (if defined and what the deprecated method doing)

Maybe it needs to file another issue to review current usage of temp-dir and tempnam() to make all usages to use service instead of sys_get_temp_dir() like

core/lib/Drupal/Core/Database/Driver/sqlite/Install/Tasks.php:79: $connection_info['default']['database'] = \Drupal::service('file_system')->tempnam(sys_get_temp_dir(), 'sqlite');
and
core/tests/Drupal/KernelTests/Core/Theme/FrontMatterTest.php:63: $file = tempnam(sys_get_temp_dir(), 'twig') . ".html.twig";

  1. +++ b/core/lib/Drupal/Core/Test/TestDatabase.php
    @@ -153,7 +152,7 @@ public function releaseLock() {
    +    $tmp = sys_get_temp_dir();
    
    @@ -176,7 +175,7 @@ public static function releaseAllTestLocks() {
    +    return sys_get_temp_dir() . '/test_' . $lock_id;
    
    +++ b/core/tests/Drupal/Tests/Scripts/TestSiteApplicationTest.php
    @@ -335,7 +334,7 @@ protected function addTestDatabase($db_prefix) {
    -    return FileSystem::getOsTemporaryDirectory() . '/test_' . $lock_id;
    +    return sys_get_temp_dir() . '/test_' . $lock_id;
    

    I wonder why OS temp dir (sys_get_temp_dir) used in tests, I think all of them should be site-specific so should use temp-dir from file system service.

    Moreover checksing current usage of sys_get_temp_dir() in core codebase it mostly used in tests and few places for install

  2. +++ b/core/modules/system/system.install
    @@ -685,7 +684,7 @@ function system_requirements($phase) {
           // If the temporary directory is not overridden use an appropriate
           // temporary path for the system.
    -      $directories[] = FileSystemComponent::getOsTemporaryDirectory();
    +      $directories[] = sys_get_temp_dir();
    

    here the else branch could be removed as system should not care about OS-temp because at runtime it supposed to use service to determine actually used dir (less code in install file)

daffie’s picture

Status: Needs review » Needs work

+1 for just deprecating the method.

+++ b/core/lib/Drupal/Component/FileSystem/FileSystem.php
@@ -13,34 +13,15 @@ class FileSystem {
-    $directories = [];
-
-    // Has PHP been set with an upload_tmp_dir?
-    if (ini_get('upload_tmp_dir')) {
-      $directories[] = ini_get('upload_tmp_dir');
-    }
-
-    // Operating system specific dirs.
-    if (substr(PHP_OS, 0, 3) == 'WIN') {
-      $directories[] = 'c:\\windows\\temp';
-      $directories[] = 'c:\\winnt\\temp';
-    }
-    else {
-      $directories[] = '/tmp';
-    }
-    // PHP may be able to find an alternative tmp directory.
-    $directories[] = sys_get_temp_dir();
-
-    foreach ($directories as $directory) {
-      if (is_dir($directory) && is_writable($directory)) {
-        // Both sys_get_temp_dir() and ini_get('upload_tmp_dir') can return paths
-        // with a trailing directory separator.
-        return rtrim($directory, DIRECTORY_SEPARATOR);
-      }
-    }
-    return FALSE;
...
+    return sys_get_temp_dir();

I think that changing what the method does might result in a BC break. Just doing a deprecation is enough.

andypost’s picture

Status: Needs work » Needs review
StatusFileSize
new1.41 KB
new8.93 KB

Fix for #21

I see no much difference if sys_get_temp_dir() will return trailing slash but I think about follow-up to use file-system service defined temp-dir for tests consistently

andypost’s picture

StatusFileSize
new549 bytes
new9.46 KB

Clean-up one more mention

Status: Needs review » Needs work

The last submitted patch, 23: 3063704-23.patch, failed testing. View results

andypost’s picture

Status: Needs work » Needs review
StatusFileSize
new1.13 KB
new10.59 KB

Fix test and one more usage

daffie’s picture

Issue summary: View changes
Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs issue summary update

The method has been deprecated.
Deprecation message test has been added.
All use of the code has been replaced.
It is not used in documentaton.
The testbot is happy.
The Is and the CR are in order.
For meit is RTBC.

berdir’s picture

I'm still uncertain about this, see #18. Just not removing the code in the existing method doesn't really address my concerns IMHO.

Are we _certain_ that those checks for the ini setting and windows aren't required anymore? Did we test the impact of this patch on Windows?

What exactly do we gain from removing and deprecating this code? Is it worth the risk of possibly breaking someones hosting setup?

catch’s picture

Status: Reviewed & tested by the community » Needs review
Issue tags: +needs manual testing on Windows

Let's get this manually tested on windows.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

Status: Needs review » Needs work
Issue tags: +Needs Review Queue Initiative

Unfortunately missed the D10 window

And still needs windows testing. (Can't do as I'm on a mac)

Akhil Yadav’s picture

StatusFileSize
new9.43 KB

Added patch against #26 in 10.1 version

andypost’s picture

+++ b/core/lib/Drupal/Component/FileSystem/FileSystem.php
@@ -13,8 +13,14 @@ class FileSystem {
+   * @deprecated in drupal:9.3.0 and is removed from drupal:10.0.0. Use
...
+    @trigger_error(__METHOD__ . ' is deprecated in drupal:9.3.0 and is removed from drupal:10.0.0. Use sys_get_temp_dir() instead. See https://www.drupal.org/node/3225275', E_USER_DEPRECATED);

message should be changed to 10.1.0 and 11.0.0

ravi.shankar’s picture

StatusFileSize
new9.43 KB
new1.18 KB

Made changes as per comment #35.

andypost’s picture

See my patch in #26

+++ b/core/tests/Drupal/Tests/Component/FileSystem/LegacyFileSystemTest.php
@@ -0,0 +1,28 @@
+  public function testDeprecatedGetOsTemporaryDirectory() {

this test is missing in last reroll

Bhanu951 made their first commit to this issue’s fork.

bhanu951’s picture

Status: Needs work » Needs review

Raised a MR against 10.x branch from the patch in #26, addressed #35 and #37.

bramdriesen’s picture

Came here for something else, decided to do a code review instead. 😇

Merge request and changes looks good! @Bhanu951

Still needs review for the manual Windows testing mentioned in #33 also on a Mac/Linux here.

smustgrave’s picture

Asked around and can't find anyone on a windows to do testing.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new90 bytes

The Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

bramdriesen’s picture

I have a Windows VM on which I could test it. Although it's not the "real deal" of course.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

alexpott’s picture

Status: Needs work » Closed (won't fix)

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.