Today at OSCON, MySQL co-founder Michael “Monty” Widenius and I presented the “State of the Dolphin” lightning talk.

My slides for this preso weren’t too graphic, which makes them all the easier to reuse in this blog:

Use our new software!

• Use MySQL 5.1, it‘s soon going RC
• Use Falcon, it‘s soon going Beta (new transactional storage engine, faster than InnoDB on large servers)
• Use MySQL Workbench (ER Tool), Now Beta
• Use MySQL Proxy, just released
• PHPers: Use mysqlnd (Native Driver)

Go test MySQL 5.1!

• We‘re happy with the quality
  • More stable than 5.0 was four months after GA
  • RC happening very soon, GA within a few versions after that
  • A better MySQL 5.0 (thousands of small fixes)
• We‘re happy with the new functionality
  • Table / Index Partitioning
  • Row-based replication – Transfers data instead of commands
  • Full-text indexing parser plugins – Flexible full text search
  • Disk-based Data Support for MySQL Cluster
  • Replication Support for MySQL Cluster
  • XPath Support - helps any customer wanting to better navigate and search XML documents stored in MySQL
  • Internal Task Scheduler (Events)

Other goodies coming soon (5.2, 6.0, …)

• Global Backup API
• Falcon and Maria (MyISAM++) storage engine
• Further new storage engines
• Hash & Merge joins (faster subselects)
• Federated tables over ODBC
• Foreign key support for all engines

Participate in our Development!

• Report bugs! Test them! Submit patches!
• Hang out on Freenode IRC #mysql-dev
• Subscribe to commits@lists.mysql.com to see our code reviews
• Attend MySQL University, the foremost education for MySQL developers (of C/C++ code, not apps)
• MySQL Forge: List your MySQL apps! Upload your code snippets! Fix the missing documents!
• Go visit MySQL Forge Worklog
  • Voting for best features starting soon

Hiring!

We are hiring outstanding C/C++ developers with systems or database engineering experience.
Visit www.mysql.com/jobs or email resume to jpugh@mysql.com

MySQL is indebted to the Software Freedom Law Center for very good advice and insight on how to combine Free Software with a viable business model. SFLC provides legal representation and other law-related services to protect and advance Free and Open Source Software. Founded in 2005, the Center now represents many of the most important and well-established free software and open source projects.

Professor Eben Moglen, SFLC director and FSF legal counsel, has provided us with profound guidance over the years. We have tried to give something back through our work in the GPLv3 Committee B, but our time resources as a small company are limited in comparison to our fellow committee members.

In recognition of Eben’s help and as a token of our appreciation, we’ve made a small donation to support the continued work of the SFLC. We encourage others who build their business on free software to do the same.

Thank you, Eben!

Finally!

We have a release of MySQL Community Server that contains community-provided contributions and it is ready for your download in binary form.

Thanks to Jeremy Cole for providing us with the SHOW PROFILE patch. Quoting our Reference Manual at http://dev.mysql.com/doc/refman/5.0/en/show-profiles.html:

SHOW PROFILES

SHOW PROFILE [type [, type] … ]
[FOR QUERY n]
[LIMIT n [OFFSET n]]

type:
ALL | BLOCK IO | CONTEXT SWITCHES | CPU | IPC | MEMORY | PAGE FAULTS | SOURCE | SWAPS

The SHOW PROFILES and SHOW PROFILE statements display profiling information that indicates resource usage for statements executed during the course of the current session.

Profiling is controlled by the profiling session variable, which has a default value of 0 (OFF). Profiling is enabled by setting profiling to 1 or ON:

mysql> SET profiling = 1;
SHOW PROFILES displays a list of the most recent statements sent to the master. The size of the list is controlled by the profiling_history_size session variable, which has a default value of 15. The maximum value is 100. Setting the value to 0 has the practical effect of disabling profiling.

All statements are profiled except SHOW PROFILES and SHOW PROFILE, so you will find neither of those statements in the profile list. Malformed statements are profiled. For example, SHOW PROFILING is an illegal statement, and a syntax error occurs if you try to execute it, but it will show up in the profiling list.

SHOW PROFILE displays detailed information about a single statement. Without the FOR QUERY n clause, the output pertains to the most recently executed statement. If FOR QUERY n is included, SHOW PROFILE displays information for statement n. The values of n correspond to the Query_ID values displayed by SHOW PROFILES.

The LIMIT n clause may be given to limit the output to n rows. If LIMIT is given, OFFSET n may be added to begin the output n rows into the full set of rows.

Some sample output:

mysql> SHOW PROFILES;
+----------+----------+--------------------------+
| Query_ID | Duration | Query                    |
+----------+----------+--------------------------+
|        0 | 0.000088 | SET PROFILING = 1        |
|        1 | 0.000136 | DROP TABLE IF EXISTS t1  |
|        2 | 0.011947 | CREATE TABLE t1 (id INT) |
+----------+----------+--------------------------+
3 rows in set (0.00 sec)

mysql> SHOW PROFILE;
+----------------------+----------+
| Status               | Duration |
+----------------------+----------+
| checking permissions | 0.000040 |
| creating table       | 0.000056 |
| After create         | 0.011363 |
| query end            | 0.000375 |
| freeing items        | 0.000089 |
| logging slow query   | 0.000019 |
| cleaning up          | 0.000005 |
+----------------------+----------+
7 rows in set (0.00 sec)

This information is also visible in the corresponding INFORMATION_SCHEMA.PROFILING table.

Unfortunately the SHOW PROFILE implementation as of 5.0.37 still has a small gotcha, that we will resolve for the next release: the MySQL command line client hangs if you issue the SHOW PROFILE command without having enabled the @@profiling session variable before. Hence, when you evaluate this new feature, make sure to enable it by issuing “SET @@profiling = 1;“ before.

In summary, the SHOW PROFILE patch provides a way for the MySQL Community Server users to get a more detailed understanding of where the response time is formed. Jeremy also contributed a patch to display the Uptime_since_flush_status status variable, which indicates the number of seconds since the most recent FLUSH STATUS statement. Thanks, Jeremy!

Looking beyond the community enhancements, MySQL Community Server 5.0.37 is our second full (source and binary) release of the MySQL Community Server since we made the split between the Community and Enterprise Version in October 2006. It includes all bug fixes applied to up to and including the MySQL 5.0.36 Enterprise Server. The prior MySQL Community Server 5.0.33 release early January 2007 was a source-only release.

The 5.0.37 release also resolves a crashing bug that could be exploited as a potential local Denial of Service attack. Thank you, Stefan Streichsbier from SEC Consult, who also informed us about this bug via our security@mysql.com mail address - he will send out a separate advisory about his findings.

We welcome and appreciate your feedback, bug reports, bug fixes, patches etc. as described on MySQL Forge on http://forge.mysql.com/wiki/Contributing.

Contrary to some reports in the community, MySQL will continue providing binaries both for Windows and other operating systems. All our download pages, including those for MySQL 5.0, have binaries today, and will continue to have them.

The source-only releases we introduced with 5.0.33 (and will continue to provide in the future)are just in addition to the binary-and-source releases. The current latest binary-and-source MySQL Community Server release is 5.0.27, and I expect MySQL 5.0.35 Community Server to be released as binary-and-source within a month, both for Windows and our other platforms. This is as we always planned it, and tried to communicate it. I am sorry our communication has not been clear enough.

MySQL Community Server 5.0.33 has now been released. It is a pure bugfix release, delivered in a source-only form as a tarball for Unix and for Windows (we provide separate sources for these, as the build procedure differs) on dev.mysql.com/downloads/mysql/5.0.html#Source.

The release contains all bug fixes applied to MySQL Server 5.0 since the last Community release 5.0.27 in October 2006. The jump in numbers from .27 to .33 is to make it clear that .33 is up-to-date to the level of MySQL Enterprise Server 5.0.32, released about two weeks ago. We reserve the even version numbers for MySQL Enterprise, while odd version numbers indicate a Community Server Release. Both use the same “5.0″ version part, as they share the same code base.

We are in the process of applying several patches provided by Community members, currently in our Contributions Pipeline. I expect several of these enhancements to be merged soon enough for the next MySQL Community Server to be released in February as 5.0.35. On the contrary, they will not appear in MySQL Enterprise Server 5.0.

The release notes for MySQL 5.0 have been split into separate sections of the MySQL 5.0 reference manual. This means that the documentation of the bug fixes of MySQL Enterprise Server 5.0.28, 5.0.30 and 5.0.32 have been copied into the release notes for MySQL Community Server 5.0.33.

The purpose of this MySQL Community Server release is twofold:

a) to deliver recent, fresh bug fixes to the MySQL community
b) to establish and test the practice of source tarball releases

As such, the bug fixes have been available in source form both as part of the BitKeeper source repository on mysql.bkbits.net and as MySQL Enterprise Server source releases on our ftp site ftp.mysql.com. However, we believe Linux distributions and Open Source fluent community users of MySQL are better served by a tagged MySQL Community Server release, which is available on our download server and has one single set of release notes.

Through establishing source tarball releases, we follow the tradition of many other FOSS projects. This provides more possibilities for Community Contributions in the means of binary builds.

We refer to our reference manual, especially the chapter 2.4.14. MySQL Installation Using a Source Distribution when it comes to building MySQL Community Server. At the same time, I want to point out that the service of providing MySQL Enterprise Server binaries is something we do for our paying customers, in the form of the MySQL Enterprise Server subscription, starting at 595 dollars a year.

We strive to release MySQL Enterprise Server on a monthly basis. While we don’t have a specific schedule or policy for when MySQL Community Server is released in binary form, I expect the next Community release, 5.0.35, to be available as source and binaries for the same platforms as MySQL Enterprise Server and as the previous MySQL Community Server binary release 5.0.27. Until that point in time, the 5.0.27 binaries will be the ones listed on the normal MySQL 5.0 download pages at dev.mysql.com/downloads/mysql/5.0.html.

Distributors of MySQL Community Server are welcome to contact Colin Charles and Lenz Grimmer, with proposals for process improvements (at firstname@mysql.com. Better still, subscribe to the packagers@lists.mysql.com archived and public mailing list at lists.mysql.com/packagers, which Colin and Lenz are monitoring. That list as our discussion forum for builders of MySQL releases, both already existing ones like the various distributors, and for new community contributors that want to step up and provide binaries, including Windows.

Please note that while the release has undergone testing, 5.0.33 is our first source only MySQL Community Server release, so I expect there to be some room for process improvement on our side.

Amongst the bug fixes, there is one that I want to highlight. It is listed as the innocent-looking bullet “InnoDB showed substandard performance with multiple queries running concurrently. (Bug#15815)“. This is a fix to an issue especially surfacing on multiple-core processors. You testing in such environments is much appreciated. As a reference, you may want to read the Bugs database entry bugs.mysql.com/15815 as well as two articles from Peter Zaitsev — one from a week ago, another from last September.

As I have seen some concerns about the release schedule of MySQL Community Server and the availability of MySQL Enterprise Server sources, let me recap and detail some of the plans, which haven’t changed since the introduction of MySQL Enterprise Server in October:

1. MySQL 5.0 Community Server sources and binaries are available from our download pages. The latest version is 5.0.27, released in October.
2. MySQL Enterprise Server is released more frequently than MySQL Community Server. This means that we’ve seen also 5.0.28, 5.0.30 and 5.0.32 being released for MySQL Enterprise Server. Even numbers are for Enterprise, odd numbers for Community.
3. MySQL Community Server gets all bug fixes from MySQL Enterprise Server. This means that the next MySQL Community Server will contain all bug fixes from the most recent MySQL Enterprise Server published at that point. If we’re releasing a MySQL Community Server right after MySQL Enterprise Server 5.0.34, it will be labeled 5.0.35 and contain all 5.0.34 fixes plus any community patches applied at that point in time.
4. The higher release frequency of MySQL Enterprise Server provides added value for our commercial customers. We intend to deliver predictable monthly Enterprise releases. Providing and verifying frequent binaries is a paid-for service for those who want to spent money to save time. That said, MySQL continues to provide new community binaries from time to time, as said in item 1 above.
5. MySQL Community Server additionally includes what we call Community Enhancements on top of MySQL Enterprise Server. However, we’ve been slow at applying these enhancements. Jeremy Cole and others have been contributing patches, and we are looking at getting several of his enhancements and those of others into the tree ASAP. We are also looking at improving our processes so that contributors won’t have to wait as long in the future.
6. MySQL Enterprise Server is available in source form for download from our ftp server at ftp.mysql.com. We don’t highlight this much, as we would like to encourage our community users — including Linux distributions — to use our community server.
7. The MySQL Community Server tree is updated frequently with the bug fixes from the Enterprise tree. Our users can access both Community and Enterprise server from BitKeeper. However, we expect most users to want to work from tagged, numbered releases, no matter whether they use Community or Enterprise server.

One area where we feel the need to improve is the release frequency of source tarballs from the Community Tree. This will alleviate some of the burden put on a couple of Community users. Our current plan is to have the next MySQL Community Server source tarball release happen late January or early February.

MySQL 4.0 has now reached the final stage of its active development life cycle. As I noted on 12 July 2006 when announcing the MySQL Lifecycle Policy, keeping legacy versions of our software alive is expensive and time-consuming. While we know that database administrators hate to upgrade their databases, we believe that the vast majority of our customers and our community are better served by us focusing our attention on newer releases. However, we don’t want to abandon users of our older products — so we are asking them to help subsidise the cost themselves.

This means that those who wish to continue to receive our support for MySQL 3.23 and MySQL 4.0 will need to be covered by a MySQL Enterprise subscription, starting 1 Oct 2006 for MySQL 4.0.

In practice,

1. We are no longer producing new binary updates for MySQL 4.0 for the
   general public.
2. Soon, we will be removing the archived versions of old MySQL 4.0
   binaries from our web site.
3. The MySQL 4.0 source code will remain available on our web site in tarball format.

As a general rule, we encourage everyone to upgrade from MySQL 4.0 to a newer release.

However, those who wish to continue to get support for MySQL 4.0 during its Extend Lifecycle Timeframe (which will continue for another two years until 31 Dec 2008) are referred to the MySQL Enterprise subscription service.

A MySQL Enterprise subscription will not only give you access to support for older versions, but also assistance with migrating to the most current production release, MySQL 5.0.

Today, you will see an announcement of a new flagship commercial offering from our company, called MySQL Enterprise. I want to explain to you why we are making these changes to our business — and to the delivery of our software.

We recognise that the needs of the MySQL Community are different from the needs of commercial enterprise customers. After 11 years of producing our software, we can no longer hope that a single offering is the best solution for both Community and Enterprise users. Consequently, we are introducing two different offerings for each distinct target group.

The MySQL Community Server is: for the Open Source fluent audience, do-it-yourself (DIY) for those who don’t need support free-of-charge

The MySQL Enterprise Server is: for the non-DIY commercial user part of the ‘MySQL Enterprise’ subscription offering for those who want extra help developing, deploying and managing MySQL DBs coupled with access to MySQL technical support assisted by new automated DBA monitoring and advisory services

With this differentiation, we aim to better serve both categories of MySQL users — those who are willing to spend time to save money, and those who are willing to spend money to save time.

If our changes succeed in their objective, both audiences will benefit from a more stable, feature-rich and high-quality database. The open source benefits for each of the audiences mutually reinforce each other:

• Community users get new features at no-cost to them — funded by paying customers
• Enterprise users get a more stable, reliable and predictably-released product — thanks to community participation

Each of these components of the virtuous circle of open source contributes to the development and spreading of a better MySQL for everyone.

By the name MySQL Enterprise, we want to make clear that this is the offering we expect business users of MySQL to be interested in: Are you using MySQL in a production enterprise setting? Go for MySQL Enterprise!

We believe the users of MySQL Community Server expect

• early access to MySQL features under development
• that MySQL AB will listen to their input
• timely corrections to bug fixes they report
• help with enhancing MySQL for their particular needs
• channels to communicate with the rest of community for getting assistance
• an easier process for having contributions accepted in MySQL
• commitment to Open Source — including free, unrestricted availability of source code

and this is what we will continue to deliver.

We’re happy to note the growth in contributions flowing into MySQL and its ecosystem. To facilitate these, we have

• launched MySQL Forge (see forge.mysql.com)
• established a Contribution License Agreement (see MySQL_Contributor_License_Agreement on Forge Wiki)
• supported a MySQL Community Camp (see mysqlcamp.org)
• started to Doxygen comment our code for easier understandability (see CommunityDoxygenProject on Forge Wiki)

and this is now showing results:

• 159 Forge projects by 58 distinct contributors
• 44 Forge snippets by 25 distinct contributors
• 361 forge users registered
• 1696 distinct Forge Wiki page titles
• 184 Forge wiki contributors
• MySQL Server kernel contributions accepted, such as Jeremy Cole’s

On top of this, we expect to soon launch a competition for voting on the look-and-feel of our new MySQL Community Server logotype and Web site. We will also be establishin a pilot program for MySQL Quality Assurance contributors. Most importantly, we’re launching the MySQL Winter of Code program, featuring the Connectors Contest and the Storage Engines Encounters, which I will be telling you about separately next week.

Technically, the MySQL Enterprise Server inherits the current MySQL 5.0.26 code base, as does the MySQL Community Server. However, we will be encouraging and incorporating contributions in the form of minor enhancements and experimental features already into the 5.0 version of MySQL Community Server. This way, contributors don’t have to wait until the next major release for their improvements to get into use, and enterprise users can continue using 5.0 without seeing any destabilisation of the code base due to new functionality being introduced.

As part of our differentiation, we will do more frequent binary releases of the MySQL Enterprise Server software than of the MySQL Community Server. However, all of our database software is open source, so we will continue to make all releases available over our BitKeeper tree and as source code tarballs — even if the MySQL Enterprise Server binaries will not be available for public download but limited to our commercial customers and our core QA contributors.

Finally, we will continue to be active good citizens in the greater Free and Open Source Software world. We’re participating in the GPLv3 drafting process, we’re supporting the Free Software Foundation as FSF corporate patrons, and we’re supporting campaigns against the spread of software patents around the globe.

So: Click here to see today’s press releases on MySQL Enterprise, which describes our new flagship commercial offering, directed at paying enterprise customers. It refers to the MySQL Network Monitoring & Advisory Services, which is a commercial only offering we are about to launch. In due course, I’ll be sharing more about that with you. Stay tuned, and please give me your feedback on what you think (including private email to k a j at m y s q l . c o m)!

Database administrators hate to upgrade their databases. At MySQL, we like to think that we have been early to recognise this, and we have given more or less unlimited support to even very old releases, on a multitude of platforms. However, this has not been without cost to ourselves.

Many of our users know that the cost of maintaining several releases is high. We have thus been asked to clarify our support lifecycle policy. After long internal discussions, that were not always easy, we are now pleased to say that we have an explicit support lifecycle policy. It addresses the timeframes we will provide updates and continued support for current and older versions of the MySQL server.

Keeping older versions alive for a long time is appreciated by our community and our customers alike. However, we are no longer in a position to maintain our older versions without remuneration. This means that those wishing to enjoy MySQL’s support in their usage of MySQL 3.23 and MySQL 4.0 need to plan for their future, as our support of these releases will be limited to those covered by a MySQL Network subscription, starting in 1 Aug 2006 (for MySQL 3.23) and 1 Oct 2006 (for MySQL 4.0).

This means that we are no longer offering publicly available binary updates for our 3.23 and 4.0 releases. The sources of these will still be provided. We are also evaluating whether we will continue hosting archived versions of old binaries, as well as the timeline for the support of MySQL 4.1.

We hope the clarity offered by this explicit support lifecycle policy will give a good base to our Customers and Users alike, when planning for their future use of MySQL!

Dear user of MySQL,

It is my pleasure to announce the production release of MySQL 5.0, which is hereby GA (Generally Available). Since my announcement of the Release Candidate less than a month ago, no bugs have been reported that require a second Release Candidate. This, combined with the feedback from over two million downloads of MySQL 5.0 during its beta phase, give us the confidence to give MySQL 5.0 the status of Current Production Release, or GA.

In the Release Candidate announcement less than a month ago, I described MySQL 5.0 as “the most important release in MySQL’s history”, and that is certainly the case. Thus, I encourage you all to:

• get your own copy at http://dev.mysql.com/downloads/mysql/5.0.html
• do all of your new database development using MySQL 5.0
• upgrade your current MySQL environments to MySQL 5.0, as soon as you’ve properly verified your production applications against it (be sure to take a full backup of your data before upgrading, study the relevant documentation, and if you have a MySQL Network support contract, consult first with the MySQL Support Team)

Let me also underline that we continue to offer some earlier versions of MySQL Server for download. However, you should expect maintenance releases for earlier versions only in limited form:

• for MySQL 4.1, only when serious bugs affecting significant user groups are reported
• for MySQL 4.0, only when security bugs are reported

MySQL 5.0 is the most ambitious release to date for MySQL AB. We have added functionality that our users have requested from us over many years. However, everything we do at MySQL centers around our three priorities of Performance, Reliability, and Ease of Use. MySQL 5.0 is certainly true to these company-wide values.

Key new features of MySQL 5.0 come in three groups:

1. ANSI SQL standard features formerly unknown to MySQL
2. ANSI SQL standard compliance of existing MySQL features
3. New MySQL Storage Engines, Tools and Extensions

1. The new ANSI SQL features include:

• Views (both read-only and updatable views)
• Stored Procedures and Stored Functions, using the SQL:2003 syntax, which is also used by IBM’s DB2
• Triggers (row-level)
• Server-side cursors (read-only, non-scrolling)

2. Implementing ANSI SQL standard ways of using existing MySQL features means there will be fewer unpleasant surprises (”gotchas”) for those migrating to MySQL from other database systems:

• Strict Mode: MySQL 5.0 adds a mode that complies with standard SQL in a number of areas in which earlier versions did not; we now do strict data type checking and issue errors for all invalid dates, numbers and strings as expected
• INFORMATION_SCHEMA: An ANSI SQL-compliant set of tables that provide database metadata, in parallel with the MySQL-specific SHOW commands
• Precision Math: A new library for fixed-point arithmetic, giving high accuracy for financial and mathematical operations
• VARCHAR Data Type: The maximum effective length of a VARCHAR column has increased to 65,532 bytes; also, stripping of trailing whitespace no longer occurs

3. New MySQL Storage Engines, Tools and Extensions are:

• XA Distributed Transactions
• ARCHIVE Storage Engine for storing large amounts of data without
  indexes in a very small footprint, intended for historical data that
  may be needed for future audit compliance (Sarbanes Oxley or
  otherwise)
• FEDERATED Storage Engine for accessing data ín tables of remote
  databases rather than in local tables (only in MAX version)
• Instance Manager: a tool to start and stop MySQL Server, even remotely

To find out more details on what’s new in MySQL 5.0, follow the pointers from http://dev.mysql.com/doc/mysql/en/mysql-5-0-nutshell.html

To find out the changes specific to MySQL 5.0.15 in relation to 5.0.13 (the release candidate), see the two files http://dev.mysql.com/doc/mysql/en/news-5-0-14.html and http://dev.mysql.com/doc/mysql/en/news-5-0-15.html (5.0.14 was not released publicly).

MySQL 5.0 is also reflected in our GUI tools and Connectors:

MySQL Administrator 1.1.4 and MySQL Query Browser 1.1.17 are aware of the new MySQL 5.0 features. They can be used to write and test stored procedures, create views, include them in scheduled backups and much more.

The latest shipping versions of our Connectors work with MySQL 5.0, and all connectors (MySQL Connector/ODBC, Connector/J and Connector/NET) support all MySQL 5.0 flagship features.

Of course, we recognize that any piece of software contains bugs. We continue to need your involvement to ensure that MySQL 5.0 is the best that it possibly can be. Should you find any issues in MySQL 5.0, report them through our bug-reporting system at http://bugs.mysql.com/ and we will improve upon MySQL 5.0 in upcoming maintenance releases.

The MySQL team looks forward to your input

• in our MySQL Forums at http://forums.mysql.com/
• in the bug database at http://bugs.mysql.com/
• in our mailing lists at http://lists.mysql.com/
• in the PlanetMySQL blog aggregation via http://www.planetmysql.org/newfeed.php
• in the User Comments of our manual at http://dev.mysql.com/doc/mysql/en/index.html (specifically for Documentation comments)
• and in the form of downloads from http://dev.mysql.com/downloads/mysql/5.0.html

MySQL 5.0 is available now. Go download it, install it, and take benefit from its many new features.

And do keep us informed on how MySQL can help support you!

Kaj Arnö
VP Community Relations
MySQL AB