With every version, the number of PostgreSQL features supported by YugabyteDB inevitably increases. This is good! However, it is pretty difficult to come by a comprehensive list of unsupported features. There’s a roadmap on GitHub[1], but the roadmap lists top-level features and misses the exact list of statements and keywords. There are also GitHub issues, but there’s over 11k of them so it could be pretty difficult to filter out what’s really unsupported.
I have been recently looking at exactly that problem while preparing the end-user documentation for one of the projects where YugabyteDB plays the core role.
As it turns out, there’s a very simple method to learn what’s missing. PostgreSQL contains a YACC grammar file, and yugabyteDB contains the same-but-slightly-modified gram.y file[2].
We can learm some interesting things from the grammar, for example these YugabyteDB specific definitions:
#define parser_yyerror(msg) scanner_yyerror(msg, yyscanner)
#define parser_errposition(pos) scanner_errposition(pos, yyscanner)
#define parser_ybc_not_support(pos, feature) \
ybc_not_support(pos, yyscanner, feature " not supported yet", -1)
#define parser_ybc_warn_ignored(pos, feature, issue) \
ybc_not_support_signal(pos, yyscanner, feature " not supported yet and will be ignored", issue, WARNING)
#define parser_ybc_signal_unsupported(pos, feature, issue) \
ybc_not_support(pos, yyscanner, feature " not supported yet", issue)
#define parser_ybc_not_support_in_templates(pos, feature) \
ybc_not_support_in_templates(pos, yyscanner, feature " is not supported in template0/template1 yet")
#define parser_ybc_beta_feature(pos, feature, has_own_flag) \
check_beta_feature(pos, yyscanner, has_own_flag ? "FLAGS_ysql_beta_feature_" feature : NULL, feature)
These definitions are used further in the grammar to annotate which features are not yet implemented. The ones which are of the most interest to us are:
parser_ybc_not_support: indicates an unsupported feature, no tracking issue,parser_ybc_warn_ignored: indicates an unsupported and ignored feature, these features generate warnings during query execution but do not result in errors, the issue argument identifies a GitHub issue used to track the feature,parser_ybc_signal_unsupported: indicates an unsupported feature, these features result in errors during query execution, the issue argument identifies a GitHub issue used to track the feature.
With this input, we can rather quickly get a good overview of what’s not yet in YugabyteDB.
And, we can do it for each individual version. Let’s try for 2.11.1:
|
|
Produces output similar to:
#define parser_ybc_not_support(pos, feature) \
| AlterObjectDependsStmt { parser_ybc_not_support(@1, "This statement"); }
| AlterSystemStmt { parser_ybc_not_support(@1, "This statement"); }
| AlterCompositeTypeStmt { parser_ybc_not_support(@1, "This statement"); }
| AlterPublicationStmt { parser_ybc_not_support(@1, "This statement"); }
| AlterSubscriptionStmt { parser_ybc_not_support(@1, "This statement"); }
| AlterTSConfigurationStmt { parser_ybc_not_support(@1, "This statement"); }
| AlterTSDictionaryStmt { parser_ybc_not_support(@1, "This statement"); }
| ClusterStmt { parser_ybc_not_support(@1, "This statement"); }
| CreateAmStmt { parser_ybc_not_support(@1, "This statement"); }
| CreateAssertStmt { parser_ybc_not_support(@1, "This statement"); }
| CreateConversionStmt { parser_ybc_not_support(@1, "This statement"); }
| CreateMatViewStmt { parser_ybc_not_support(@1, "This statement"); }
| CreatePublicationStmt { parser_ybc_not_support(@1, "This statement"); }
| CreatePLangStmt { parser_ybc_not_support(@1, "This statement"); }
| CreateSubscriptionStmt { parser_ybc_not_support(@1, "This statement"); }
| CreateStatsStmt { parser_ybc_not_support(@1, "This statement"); }
| CreateTransformStmt { parser_ybc_not_support(@1, "This statement"); }
| DropAssertStmt { parser_ybc_not_support(@1, "This statement"); }
| DropPLangStmt { parser_ybc_not_support(@1, "This statement"); }
| DropSubscriptionStmt { parser_ybc_not_support(@1, "This statement"); }
| DropTransformStmt { parser_ybc_not_support(@1, "This statement"); }
| RefreshMatViewStmt { parser_ybc_not_support(@1, "This statement"); }
| LoadStmt { parser_ybc_not_support(@1, "This statement"); }
| ReindexStmt { parser_ybc_not_support(@1, "This statement"); }
| SecLabelStmt { parser_ybc_not_support(@1, "This statement"); }
parser_ybc_not_support(@6, "CREATE SCHEMA with elements");
parser_ybc_not_support(@4, "CREATE SCHEMA with elements");
parser_ybc_not_support(@1, "CREATE STATISTICS");
parser_ybc_not_support(@1, "CREATE STATISTICS");
parser_ybc_not_support(@1, "CREATE LANGUAGE");
parser_ybc_not_support(@1, "CREATE LANGUAGE");
parser_ybc_not_support(@1, "DROP LANGUAGE");
...
Okay, so we can use the same method to search for:
- ignored features:
grep parser_ybc_warn_ignored gram-${YBDBV}.y: - unsupported features tracked in GitHub issues:
grep parser_ybc_signal_unsupported gram-${YBDBV}.y
To gain more context into surrounding code, use -B num-lines flag of grep.
§tracking progress between versions
Now that we can identify unsupported and ignored features, let’s see if we can identify progress between versions. We are going to compare 2.11.1 and 2.11.2.
|
|
The line counts are 139 and 136 respectively. Let’s see what has been removed from 2.11.2, which indicates that feature support has been added:
|
|
and compare the out files:
|
|
which produces:
|
|
This indicates that the 2.11.2 version of YugabyteDB added support for CREATE/REFRESH/DROP MATERIALIZED VIEW statements.
Indeed, if we look at the release notes for 2.11.2[3], we will find the following line: [YSQL] Add support for CREATE, DROP, and REFRESH MATERIALIZED VIEW.
Release notes also suggest that ALTER TYPE .. RENAME TO and ADD CONSTRAINT .. UNIQUE .. USING INDEX are supported in 2.11.2. Let’s try to identify those using this method:
|
|
gives:
|
|
There it is. Next to already established materialized views features, we can find new ALTER INDEX and ALTER TYPE. Let’s gain a little bit more context into, say, ALTER TYPE change.
We know that the feature has been added in 2.11.2, those lines do not exist in the output for 2.11.2 but exist in the output for 2.11.1. So we can do:
|
|
gives:
|
|
For some of those statements, it might be necessary to adjust the value of -B. In this case 5 was enough, we can see that the parser_ybc_signal_unsupported for ALTER TYPE_P any_name RENAME TO name disappeared from the grammar file in 2.11.2.
With little to no work, it’s possible to quickly compare the level of support for all three definitions across whatever arbitrary versions we are interested in.