YAML, what is it good for?
YAML (YAML Ain’t Markup Language) has been a structured data format frequently used in of Ruby development for some time. It has many useful features that often go unnoticed. Its ability to store serialized objects has recently made much news in the form of very serious vulnerabilities in the Rails framework.
Let’s explore ways of using YAML beyond a simple store of arrays and hashes of data, as well as the risks and benefits of doing so.
We will discuss YAML databases, configuration files, conversion into and from Ruby objects, incompatibilities between parsers (Syck and Psych) and the reasons for them, and some gotchas.
Some reference will be made to YAML libraries in Ruby and other languages. http://www.yaml.org/
In progress notes on the topic follow
YAML Spec and Terminology
- JSON superset
The primary objective of this revision is to bring YAML into compliance with JSON as an official subset. YAML 1.2 is compatible with 1.1 for most practical applications - this is a minor revision. An expected source of incompatibility with prior versions of YAML, especially the syck implementation, is the change in implicit typing rules. We have removed unique implicit typing rules and have updated these rules to align them with JSON’s productions. In this version of YAML, boolean values may be serialized as “true” or “false”; the empty scalar as “null”. Unquoted numeric values are a superset of JSON’s numeric production. Other changes in the specification were the removal of the Unicode line breaks and production bug fixes. We also define 3 built-in implicit typing rule sets: untyped, strict JSON, and a more flexible YAML rule set that extends JSON typing.
YAML Ruby Library Code
Psych and Syck
- 1.8 : uses Syck
- 1.9 YAML == Psych, but can YAML::ENGINE.yamler = ‘syck’
- 2.0 YAML == Psych, Syck removed from stlibi
- What this means for your startup
- YAML F7U12
- Rails’ Remote Code Execution Vulnerability Explained
- Analysis of Rails XML Parameter Parsing Vulnerability
- Reddit Discussion Of: Serious vulnerability in Ruby on Rails allowing arbitrary Ruby code execution in any Rails application (groups.google.com)
- Psych issue to consider adding #safe_load
- [SEC][ANN] Rails 3.2.12, 3.1.11, and 2.3.17 have been released!
- Multiple vulnerabilities in parameter parsing in Action Pack (CVE-2013-0156)
- Rails PoC exploits for CVE-2013-0156 and CVE-2013-0155
- Metasploit Framework: Ruby on Rails XML Processor YAML Deserialization Scanner
- Serialization Mischief in Ruby Land (CVE-2013-0156)
- Exploiting Ruby on Rails with Metasploit (CVE-2013-0156)
- Analysis of Rails XML Parameter Parsing Vulnerability and the commit that introduced it
- YAML Support and Hash Representer In representable-1.2.7!
- Putting YAML at the top of a Markdown file with Metadown
Misc dump for now
- YAML is terrible and should be driven from the face of the Earth.
- Psych 2.0
- Rails 4 uses Josh Peek’s sprockets-rails 2 manifest files are now json, not yaml. In communication: JSON has better support in non-Ruby languages.
- How RubyGems.org patched Psych/Syck following the 2013 January attacks
- Rails Allow to_xml and from_xml to work with AR models that have serialized fields
- Psych: Strings that look like dates should be treated as strings and not dates.
- Fun with Syck and Psych on Heroku. This is one of those posts that’s inspired by a desire never to have to investigate the same bug again.
- Psych-based yaml in Ruby 1.9.3 too slow
- Psych can’t parse YML in RefineryCMS that syck can
- Delayed job 3.0.1 not working, breaks on to_yaml
- Convert Syck to Psych YAML format
- [Bug #3112] require “yaml” doesn’t use psych as default
- [ruby-trunk - Feature #6163][Open] Remove syck YAML extension
- Removing Syck from ruby
- VCR: Request/Response data is saved to disk as YAML by default
- Parsing YAML 1.1 with Ruby
- Shaving A YAML Yak
- Force YAML to use syck rather than psych. Eventually delayed_job should be made compatible with both.
- work around the syck/psych confusion yaml brings to ruby 1.9.2 by trying to load psych before yaml
- SafeYAML: Support for custom deserializer?
- JRuby 1.9 problem with Syck