Skip to content

Next

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
    • Help
    • Support
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
GitLab FOSS
GitLab FOSS
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
    • Cycle Analytics
    • Insights
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Charts
    • Locked Files
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 1
    • Merge Requests 1
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Charts
  • Security & Compliance
    • Security & Compliance
    • Dependency List
  • Packages
    • Packages
    • Container Registry
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Charts
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • GitLab.org
  • GitLab FOSSGitLab FOSS
  • Issues
  • #40168

Closed
Open
Opened Nov 15, 2017 by Michael Kozono@mkozono
  • Report abuse
  • New issue
Report abuse New issue

Uploads with long filenames on MySQL fail

Summary

Uploads with long filenames on MySQL fail.

Steps to reproduce

On a GitLab instance >= v9.0, using MySQL, upload a file with a 255 character long filename to a Markdown field (issue, MR, comment).

An insert to the uploads table fails because uploads.path is varchar(255) for Rails string fields by default, and the path is the filename plus some directories.

I think this also affects appearance logo and header logo uploads but not avatars since avatar filenames get renamed to 'avatar' by the frontend.

What is the current bug behavior?

Upload dies.

What is the expected correct behavior?

Upload happens.

I believe uploads.path needs to handle at least 298 characters for the current Uploader schemes.

Max length Path kind of looks like
297 uploads/-/system/appearance/header_logo/1/255_char_filename
297 uploads/-/system/project/avatar/999999999/255_char_filename
295 uploads/-/system/group/avatar/999999999/255_char_filename
272 16_character_hex/255_char_filename
49 uploads/-/system/user/avatar/999999999/avatar.png

Relevant logs and/or screenshots

Locally on master branch on Nov 14, 2017

image

Started POST "/h5bp/html5-boilerplate/uploads" for 127.0.0.1 at 2017-11-14 19:50:51 -0800
  Project Load (0.6ms)  SELECT  `projects`.* FROM `projects` INNER JOIN `routes` ON `routes`.`source_id` = `projects`.`id` AND `routes`.`source_type` = 'Project' WHERE ((routes.path = 'h5bp/html5-boilerplate'))  ORDER BY (CASE WHEN BINARY routes.path = 'h5bp/html5-boilerplate' THEN 0 ELSE 1 END) LIMIT 1
  CACHE (0.0ms)  SELECT  `projects`.* FROM `projects` INNER JOIN `routes` ON `routes`.`source_id` = `projects`.`id` AND `routes`.`source_type` = 'Project' WHERE ((routes.path = 'h5bp/html5-boilerplate'))  ORDER BY (CASE WHEN BINARY routes.path = 'h5bp/html5-boilerplate' THEN 0 ELSE 1 END) LIMIT 1  [["source_type", "Project"]]
Processing by Projects::UploadsController#create as JSON
  Parameters: {"file.path"=>"/Users/monozok/Developer/gitlab-development-kit/gitlab/public/uploads/tmp/multipart-012013196/255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename.jpg", "file.name"=>"255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename.jpg", "file"=>#<UploadedFile:0x00007fa2cf082c98 @content_type="application/octet-stream", @original_filename="255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename.jpg", @tempfile=#<File:/Users/monozok/Developer/gitlab-development-kit/gitlab/public/uploads/tmp/multipart-012013196/255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename.jpg>>, "namespace_id"=>"h5bp", "project_id"=>"html5-boilerplate"}
  User Load (0.6ms)  SELECT  `users`.* FROM `users` WHERE `users`.`id` = 1  ORDER BY `users`.`id` ASC LIMIT 1
  CACHE (0.0ms)  SELECT  `projects`.* FROM `projects` INNER JOIN `routes` ON `routes`.`source_id` = `projects`.`id` AND `routes`.`source_type` = 'Project' WHERE ((routes.path = 'h5bp/html5-boilerplate'))  ORDER BY (CASE WHEN BINARY routes.path = 'h5bp/html5-boilerplate' THEN 0 ELSE 1 END) LIMIT 1  [["source_type", "Project"]]
  Route Load (0.4ms)  SELECT  `routes`.* FROM `routes` WHERE `routes`.`source_id` = 16 AND `routes`.`source_type` = 'Project' LIMIT 1
  Upload Load (0.5ms)  SELECT `uploads`.* FROM `uploads` WHERE `uploads`.`path` = 'c60396973cf7ca24c4c207365190b7fb/255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename.jpg'
   (0.2ms)  BEGIN
  SQL (1.5ms)  INSERT INTO `uploads` (`size`, `path`, `model_id`, `model_type`, `uploader`, `checksum`, `created_at`) VALUES (335679, 'c60396973cf7ca24c4c207365190b7fb/255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename_255_character_long_filename.jpg', 16, 'Project', 'FileUploader', 'fdb2dfdef19b5d4add6ccd575dd13be59270cd1ed2c89f93658f282635f55d4a', '2017-11-15 03:50:52')
   (0.2ms)  ROLLBACK
Completed 500 Internal Server Error in 59ms (ActiveRecord: 4.6ms)

Mysql2::Error - Data too long for column 'path' at row 1:
  mysql2 (0.4.5) lib/mysql2/client.rb:120:in `block in query'
  mysql2 (0.4.5) lib/mysql2/client.rb:119:in `query'
  peek-mysql2 (1.1.0) lib/peek/views/mysql2.rb:14:in `query_with_timing'
  activerecord (4.2.8) lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `block in execute'
  activerecord (4.2.8) lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'
  activesupport (4.2.8) lib/active_support/notifications/instrumenter.rb:20:in `instrument'
  activerecord (4.2.8) lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'
  activerecord (4.2.8) lib/active_record/connection_adapters/abstract_mysql_adapter.rb:309:in `execute'
  activerecord (4.2.8) lib/active_record/connection_adapters/mysql2_adapter.rb:225:in `execute'
  config/initializers/connection_fix.rb:20:in `execute'
  activerecord (4.2.8) lib/active_record/connection_adapters/mysql2_adapter.rb:242:in `exec_insert'
  activerecord (4.2.8) lib/active_record/connection_adapters/abstract/database_statements.rb:108:in `insert'
  activerecord (4.2.8) lib/active_record/connection_adapters/abstract/query_cache.rb:14:in `insert'
  activerecord (4.2.8) lib/active_record/relation.rb:64:in `insert'
  activerecord (4.2.8) lib/active_record/persistence.rb:524:in `_create_record'
  activerecord (4.2.8) lib/active_record/counter_cache.rb:139:in `_create_record'
  config/initializers/active_record_locking.rb:12:in `_create_record'
  activerecord (4.2.8) lib/active_record/attribute_methods/dirty.rb:133:in `_create_record'
  activerecord (4.2.8) lib/active_record/callbacks.rb:306:in `block in _create_record'
  activesupport (4.2.8) lib/active_support/callbacks.rb:88:in `__run_callbacks__'
  activesupport (4.2.8) lib/active_support/callbacks.rb:778:in `_run_create_callbacks'
  activerecord (4.2.8) lib/active_record/callbacks.rb:306:in `_create_record'
  activerecord (4.2.8) lib/active_record/timestamp.rb:57:in `_create_record'
  activerecord (4.2.8) lib/active_record/persistence.rb:504:in `create_or_update'
  activerecord (4.2.8) lib/active_record/callbacks.rb:302:in `block in create_or_update'
  activesupport (4.2.8) lib/active_support/callbacks.rb:117:in `call'
  activesupport (4.2.8) lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
  activesupport (4.2.8) lib/active_support/callbacks.rb:505:in `call'
  activesupport (4.2.8) lib/active_support/callbacks.rb:92:in `__run_callbacks__'
  activesupport (4.2.8) lib/active_support/callbacks.rb:778:in `_run_save_callbacks'
  activerecord (4.2.8) lib/active_record/callbacks.rb:302:in `create_or_update'
  activerecord (4.2.8) lib/active_record/persistence.rb:120:in `save'
  activerecord (4.2.8) lib/active_record/validations.rb:37:in `save'
  activerecord (4.2.8) lib/active_record/attribute_methods/dirty.rb:21:in `save'
  activerecord (4.2.8) lib/active_record/transactions.rb:286:in `block (2 levels) in save'
  activerecord (4.2.8) lib/active_record/transactions.rb:351:in `block in with_transaction_returning_status'
  activerecord (4.2.8) lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `block in transaction'
  activerecord (4.2.8) lib/active_record/connection_adapters/abstract/transaction.rb:184:in `within_new_transaction'
  activerecord (4.2.8) lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `transaction'
  activerecord (4.2.8) lib/active_record/transactions.rb:220:in `transaction'
  activerecord (4.2.8) lib/active_record/transactions.rb:348:in `with_transaction_returning_status'
  activerecord (4.2.8) lib/active_record/transactions.rb:286:in `block in save'
  activerecord (4.2.8) lib/active_record/transactions.rb:301:in `rollback_active_record_state!'
  activerecord (4.2.8) lib/active_record/transactions.rb:285:in `save'
  activerecord (4.2.8) lib/active_record/persistence.rb:34:in `create'
  app/models/upload.rb:22:in `record'
  app/uploaders/records_uploads.rb:21:in `record_upload'
  carrierwave (1.2.1) lib/carrierwave/uploader/callbacks.rb:16:in `block in with_callbacks'
  carrierwave (1.2.1) lib/carrierwave/uploader/callbacks.rb:16:in `with_callbacks'
  carrierwave (1.2.1) lib/carrierwave/uploader/store.rb:56:in `store!'
  app/services/upload_service.rb:12:in `execute'
  app/controllers/concerns/uploads_actions.rb:3:in `create'
  actionpack (4.2.8) lib/action_controller/metal/implicit_render.rb:4:in `send_action'
  actionpack (4.2.8) lib/abstract_controller/base.rb:198:in `process_action'
  actionpack (4.2.8) lib/action_controller/metal/rendering.rb:10:in `process_action'
  actionpack (4.2.8) lib/abstract_controller/callbacks.rb:20:in `block in process_action'
  activesupport (4.2.8) lib/active_support/callbacks.rb:117:in `call'
  activesupport (4.2.8) lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
  activesupport (4.2.8) lib/active_support/callbacks.rb:505:in `call'
  activesupport (4.2.8) lib/active_support/callbacks.rb:498:in `block (2 levels) in around'
  activesupport (4.2.8) lib/active_support/callbacks.rb:313:in `block (2 levels) in halting'
  lib/gitlab/i18n.rb:47:in `with_locale'
  lib/gitlab/i18n.rb:53:in `with_user_locale'
  app/controllers/application_controller.rb:336:in `set_locale'
  activesupport (4.2.8) lib/active_support/callbacks.rb:432:in `block in make_lambda'
  activesupport (4.2.8) lib/active_support/callbacks.rb:312:in `block in halting'
  activesupport (4.2.8) lib/active_support/callbacks.rb:497:in `block in around'
  activesupport (4.2.8) lib/active_support/callbacks.rb:505:in `call'
  activesupport (4.2.8) lib/active_support/callbacks.rb:498:in `block (2 levels) in around'
  activesupport (4.2.8) lib/active_support/callbacks.rb:298:in `block in halting_and_conditional'
  activesupport (4.2.8) lib/active_support/callbacks.rb:497:in `block in around'
  activesupport (4.2.8) lib/active_support/callbacks.rb:505:in `call'
  activesupport (4.2.8) lib/active_support/callbacks.rb:92:in `__run_callbacks__'
  activesupport (4.2.8) lib/active_support/callbacks.rb:778:in `_run_process_action_callbacks'
  activesupport (4.2.8) lib/active_support/callbacks.rb:81:in `run_callbacks'
  actionpack (4.2.8) lib/abstract_controller/callbacks.rb:19:in `process_action'
  actionpack (4.2.8) lib/action_controller/metal/rescue.rb:29:in `process_action'
  actionpack (4.2.8) lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
  activesupport (4.2.8) lib/active_support/notifications.rb:164:in `block in instrument'
  activesupport (4.2.8) lib/active_support/notifications/instrumenter.rb:20:in `instrument'
  activesupport (4.2.8) lib/active_support/notifications.rb:164:in `instrument'
  actionpack (4.2.8) lib/action_controller/metal/instrumentation.rb:30:in `process_action'
  actionpack (4.2.8) lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
  activerecord (4.2.8) lib/active_record/railties/controller_runtime.rb:18:in `process_action'
  actionpack (4.2.8) lib/abstract_controller/base.rb:137:in `process'
  actionview (4.2.8) lib/action_view/rendering.rb:30:in `process'
  actionpack (4.2.8) lib/action_controller/metal.rb:196:in `dispatch'
  actionpack (4.2.8) lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
  actionpack (4.2.8) lib/action_controller/metal.rb:237:in `block in action'
  actionpack (4.2.8) lib/action_dispatch/routing/route_set.rb:74:in `dispatch'
  actionpack (4.2.8) lib/action_dispatch/routing/route_set.rb:43:in `serve'
  actionpack (4.2.8) lib/action_dispatch/routing/mapper.rb:49:in `serve'
  actionpack (4.2.8) lib/action_dispatch/journey/router.rb:43:in `block in serve'
  actionpack (4.2.8) lib/action_dispatch/journey/router.rb:30:in `serve'
  actionpack (4.2.8) lib/action_dispatch/routing/route_set.rb:817:in `call'
  peek-performance_bar (1.3.0) lib/peek/views/performance_bar/process_utilization.rb:120:in `call'
  lib/gitlab/middleware/multipart.rb:98:in `block in call'
  lib/gitlab/middleware/multipart.rb:53:in `with_open_files'
  lib/gitlab/middleware/multipart.rb:97:in `call'
  lib/gitlab/request_profiler/middleware.rb:14:in `call'
  lib/gitlab/middleware/go.rb:18:in `call'
  flipper (0.10.2) lib/flipper/middleware/memoizer.rb:55:in `call'
  lib/gitlab/etag_caching/middleware.rb:11:in `call'
  warden (1.2.6) lib/warden/manager.rb:35:in `block in call'
  warden (1.2.6) lib/warden/manager.rb:34:in `call'
  rack-cors (0.4.0) lib/rack/cors.rb:80:in `call'
  rack-attack (4.4.1) lib/rack/attack.rb:107:in `call'
  rack (1.6.8) lib/rack/etag.rb:24:in `call'
  rack (1.6.8) lib/rack/conditionalget.rb:38:in `call'
  rack (1.6.8) lib/rack/head.rb:13:in `call'
  actionpack (4.2.8) lib/action_dispatch/middleware/params_parser.rb:27:in `call'
  lib/gitlab/middleware/read_only.rb:31:in `call'
  actionpack (4.2.8) lib/action_dispatch/middleware/flash.rb:260:in `call'
  rack (1.6.8) lib/rack/session/abstract/id.rb:225:in `context'
  rack (1.6.8) lib/rack/session/abstract/id.rb:220:in `call'
  actionpack (4.2.8) lib/action_dispatch/middleware/cookies.rb:560:in `call'
  activerecord (4.2.8) lib/active_record/query_cache.rb:36:in `call'
  activerecord (4.2.8) lib/active_record/connection_adapters/abstract/connection_pool.rb:653:in `call'
  activerecord (4.2.8) lib/active_record/migration.rb:377:in `call'
  actionpack (4.2.8) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
  activesupport (4.2.8) lib/active_support/callbacks.rb:88:in `__run_callbacks__'
  activesupport (4.2.8) lib/active_support/callbacks.rb:778:in `_run_call_callbacks'
  activesupport (4.2.8) lib/active_support/callbacks.rb:81:in `run_callbacks'
  actionpack (4.2.8) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
  actionpack (4.2.8) lib/action_dispatch/middleware/reloader.rb:73:in `call'
  actionpack (4.2.8) lib/action_dispatch/middleware/remote_ip.rb:78:in `call'
  better_errors (2.1.1) lib/better_errors/middleware.rb:84:in `protected_app_call'
  better_errors (2.1.1) lib/better_errors/middleware.rb:79:in `better_errors_call'
  better_errors (2.1.1) lib/better_errors/middleware.rb:57:in `call'
  actionpack (4.2.8) lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
  actionpack (4.2.8) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
  railties (4.2.8) lib/rails/rack/logger.rb:38:in `call_app'
  railties (4.2.8) lib/rails/rack/logger.rb:20:in `block in call'
  activesupport (4.2.8) lib/active_support/tagged_logging.rb:68:in `block in tagged'
  activesupport (4.2.8) lib/active_support/tagged_logging.rb:26:in `tagged'
  activesupport (4.2.8) lib/active_support/tagged_logging.rb:68:in `tagged'
  railties (4.2.8) lib/rails/rack/logger.rb:20:in `call'
  sprockets-rails (3.2.0) lib/sprockets/rails/quiet_assets.rb:13:in `call'
  lib/gitlab/request_context.rb:18:in `call'
  request_store (1.3.1) lib/request_store/middleware.rb:9:in `call'
  actionpack (4.2.8) lib/action_dispatch/middleware/request_id.rb:21:in `call'
  rack (1.6.8) lib/rack/methodoverride.rb:22:in `call'
  rack (1.6.8) lib/rack/runtime.rb:18:in `call'
  rack (1.6.8) lib/rack/lock.rb:17:in `call'
  actionpack (4.2.8) lib/action_dispatch/middleware/static.rb:120:in `call'
  lib/gitlab/middleware/static.rb:9:in `call'
  lib/gitlab/middleware/webpack_proxy.rb:20:in `perform_request'
  rack-proxy (0.6.0) lib/rack/proxy.rb:57:in `call'
  rack (1.6.8) lib/rack/sendfile.rb:113:in `call'
  lib/gitlab/metrics/requests_rack_middleware.rb:27:in `call'
  sentry-raven (2.5.3) lib/raven/integrations/rack.rb:50:in `call'
  railties (4.2.8) lib/rails/engine.rb:518:in `call'
  railties (4.2.8) lib/rails/application.rb:165:in `call'
  railties (4.2.8) lib/rails/railtie.rb:194:in `method_missing'
  rack (1.6.8) lib/rack/urlmap.rb:66:in `block in call'
  rack (1.6.8) lib/rack/urlmap.rb:50:in `call'
  unicorn (5.1.0) lib/unicorn/http_server.rb:562:in `process_client'
  unicorn (5.1.0) lib/unicorn/http_server.rb:658:in `worker_loop'
  unicorn (5.1.0) lib/unicorn/http_server.rb:508:in `spawn_missing_workers'
  unicorn (5.1.0) lib/unicorn/http_server.rb:132:in `start'
  unicorn (5.1.0) bin/unicorn_rails:209:in `<top (required)>'
  /Users/monozok/.rbenv/versions/2.3.5/bin/unicorn_rails:22:in `<top (required)>'
  bundler (1.16.0) lib/bundler/cli/exec.rb:75:in `kernel_load'
  bundler (1.16.0) lib/bundler/cli/exec.rb:28:in `run'
  bundler (1.16.0) lib/bundler/cli.rb:424:in `exec'
  bundler (1.16.0) lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
  bundler (1.16.0) lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
  bundler (1.16.0) lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
  bundler (1.16.0) lib/bundler/cli.rb:27:in `dispatch'
  bundler (1.16.0) lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
  bundler (1.16.0) lib/bundler/cli.rb:18:in `start'
  bundler (1.16.0) exe/bundle:30:in `block in <top (required)>'
  bundler (1.16.0) lib/bundler/friendly_errors.rb:122:in `with_friendly_errors'
  bundler (1.16.0) exe/bundle:22:in `<top (required)>'
  /Users/monozok/.rbenv/versions/2.3.5/bin/bundle:22:in `<main>'

Results of GitLab environment info

Expand for output related to GitLab environment info

(For installations with omnibus-gitlab package run and paste the output of: sudo gitlab-rake gitlab:env:info)

(For installations from source run and paste the output of: sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production)

Results of GitLab application Check

Expand for output related to the GitLab application check

(For installations with omnibus-gitlab package run and paste the output of: sudo gitlab-rake gitlab:check SANITIZE=true)

(For installations from source run and paste the output of: sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)

(we will only investigate if the tests are passing)

Possible fixes

(If you can, link to the line of code that might be responsible for the problem)

Edited Nov 15, 2017 by Michael Kozono

Related issues

  • Discussion
  • Designs
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
3
Labels
Platform [DEPRECATED] backend bug
Assign labels
  • View project labels
Reference: gitlab-org/gitlab-foss#40168