diff --git a/.gitignore b/.gitignore index d3a0169..684c0dd 100644 --- a/.gitignore +++ b/.gitignore @@ -2,3 +2,6 @@ configs test/fixtures/test*.sql pkg +.redcar +.rvmrc + diff --git a/CHANGELOG b/CHANGELOG new file mode 100644 index 0000000..d969bba --- /dev/null +++ b/CHANGELOG @@ -0,0 +1,11 @@ +0.1.1 May 3, 2011 +=== + +* updated for pg driver ~> 0.11.0 +* add a check for mysql cmd, e.g. mysql or mysql5 +* remove deprecated feature per gem -v 1.7.2 + +0.1.0 September 18, 2010 +=== + +* Initial packaging as a gem \ No newline at end of file diff --git a/README.rdoc b/README.rdoc index ae6a0dc..0c2d493 100644 --- a/README.rdoc +++ b/README.rdoc @@ -11,9 +11,8 @@ to contact me, I'll help you. Mysql2psql is packaged as a gem. Install as usual (use sudo if required). $ gem install mysql2psql - NB: the gem hasn't been formally released on http://rubygems.org/ yet. For now you need to clone the git repo at http://github.com/tardate/mysql2postgres and 'rake install' -After installation, the "mysql2psql" command will be available. +After installation, the "mysql2psql" command will be available. When run, it will generate a default configuration file in the current directory if a config file is not already found. The default configuration filename is mysql2psql.yml. @@ -42,7 +41,7 @@ After editing the configuration file and launching tool, you will see something == Database driver dependencies -Mysql2psql uses the 'mysql' and 'pg' gems for database connectivity. +Mysql2psql uses the 'mysql' and 'pg' gems for database connectivity. Mysql2psql gem installation should automatically install these too. If you encounter any issues with db connectivity, verify that the 'mysql' and 'pg' gems are installed and working correctly first. @@ -52,14 +51,25 @@ The 'mysql' gem in particular may need a hint on where to find the mysql headers NB: With Ruby 1.8.x, the gem install will usually complain about "No definition for ..." errors. These are non-fatal and just affect the documentation install. This doesn't happen with Ruby 1.9.2. +== Contributing to Mysql2psql + +* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet +* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it +* Fork the project +* Start a feature/bugfix branch +* Commit and push until you are happy with your contribution +* Make sure to add tests for it. This is important so no-one unintentionally breaks it in a future version. +* Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so the gem maintainers can cherry-pick around it. +* Send a pull request + == Getting the source -Mysql2psql was first produced by Max Lapshin who maintains the master +Mysql2psql was first produced by Max Lapshin who maintains the master repository at http://github.com/maxlapshin/mysql2postgres -The gem packaged version you are currently looking at has yet to be merged into the master repo. The gem bundling and refactoring for testing was done by Paul Gallagher and the repository is at http://github.com/tardate/mysql2postgres +Gem packaging is done with Jeweler. Note that on windows this means you must run under mingw or other shell that supports git to do the gem bundling (but you can gem install the built gem in the normal Windows shell). == Running tests If you fork/clone the project, you will want to run the test suite (and add to it if you are developing new features or fixing bugs). @@ -67,39 +77,43 @@ If you fork/clone the project, you will want to run the test suite (and add to i A suite of tests are provided and are run using rake (rake -T for more info) rake test:units rake test:integration - rake test + rake test Rake without parameters (or "rake test") will run both the unit and integration tests. Unit tests are small standalone tests of the mysql2psql codebase that have no external dependencies (like a database) -Integration tests require suitable mysql and postgres databases to be setup in advance. +Integration tests require suitable mysql and postgres databases to be setup in advance. Running the integration tests *will* rewrite data and schema for the "mysql2psql_test" databases in mysql and postgres. Don't use this database for anything else! -mysql on localhost:3306 -- database created called "mysql2psql_test" -- user setup for "mysql2psql" with no password -- e.g. +Setting up mysql on localhost:3306 +- create a database called "mysql2psql_test" +- setup a user "mysql2psql" with no password + +e.g. + mysqladmin -uroot -p create mysql2psql_test mysql -uroot -p -e "grant all on mysql2psql_test.* to 'mysql2psql'@'localhost';" # verify connecction: mysql -umysql2psql -e "select database(),'yes' as connected" mysql2psql_test - -postgres on localhost:5432 -- database created called "mysql2psql_test" -- role (user) access setup for "mysql2psql" with no password -- e.g. + + +Setting up PostgreSQL on localhost:5432 +- create a database called "mysql2psql_test" +- setup a role (user) "mysql2psql" with no password + +e.g. + psql postgres -c "create role mysql2psql with login" psql postgres -c "create database mysql2psql_test with owner mysql2psql encoding='UTF8'" # verify connection: psql mysql2psql_test -U mysql2psql -c "\c" + == Notes, Limitations, Outstanding Issues.. -Todo: +=== Todo - more tests -- release gem -- a windows cmd shim === note from mgkimsal I'm still having trouble with bit(1)/boolean fields @@ -107,11 +121,11 @@ workaround I've found is to put output in file then in VIM on file, search/replace the true/false binary fields with t/f specifically -(reversed on 3/23 - should be ^A gets f) -:%s/^@/t/g -:%s/^A/f/g -keystrokes are ctrl-v ctrl-shift-@ to get the 'true' binary field -keystrokes are ctrl-v ctrl-shift-A to get the 'false' binary field + (reversed on 3/23 - should be ^A gets f) + :%s/^@/t/g + :%s/^A/f/g + keystrokes are ctrl-v ctrl-shift-@ to get the 'true' binary field + keystrokes are ctrl-v ctrl-shift-A to get the 'false' binary field == Contributors Project founded by Max Lapshin @@ -129,7 +143,3 @@ Other contributors (in git log order): - Neszt Tibor - Miroslav Kratochvil - Paul Gallagher - - - - diff --git a/Rakefile b/Rakefile index c7193f3..c774b1d 100644 --- a/Rakefile +++ b/Rakefile @@ -30,7 +30,7 @@ begin "Paul Gallagher " ] gem.add_dependency "mysql", "= 2.8.1" - gem.add_dependency "pg", "= 0.9.0" + gem.add_dependency "pg", "~> 0.11.0" gem.add_development_dependency "test-unit", ">= 2.1.1" # gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings end diff --git a/lib/mysql2psql/config.rb b/lib/mysql2psql/config.rb index 07f6c69..6733f73 100644 --- a/lib/mysql2psql/config.rb +++ b/lib/mysql2psql/config.rb @@ -32,7 +32,7 @@ def self.template(to_filename = nil, include_tables = [], exclude_tables = [], s mysql: hostname: localhost port: 3306 - socket: /tmp/mysql.sock + socket: username: mysql2psql password: database: mysql2psql_test diff --git a/lib/mysql2psql/mysql_reader.rb b/lib/mysql2psql/mysql_reader.rb index 98dc5e8..df1e834 100644 --- a/lib/mysql2psql/mysql_reader.rb +++ b/lib/mysql2psql/mysql_reader.rb @@ -27,12 +27,16 @@ def columns def convert_type(type) case type - when /int.* unsigned/ + when /^int.* unsigned/ "bigint" when /bigint/ "bigint" when "bit(1)" "boolean" + when /smallint.* unsigned/ + "integer" + when /smallint/ + "smallint" when "tinyint(1)" "boolean" when /tinyint/ @@ -42,10 +46,12 @@ def convert_type(type) when /varchar/ "varchar" when /char/ - "char" - when /(float|decimal)/ + "char" + when /decimal/ "decimal" - when /double/ + when /float/ + "float" + when /real|double/ "double precision" else type @@ -59,6 +65,7 @@ def load_columns fields = [] @reader.mysql.query("EXPLAIN `#{name}`") do |res| while field = res.fetch_row do + length = -1 length = field[1][/\((\d+)\)/, 1] if field[1] =~ /\((\d+)\)/ length = field[1][/\((\d+),(\d+)\)/, 1] if field[1] =~ /\((\d+),(\d+)\)/ desc = { @@ -143,7 +150,14 @@ def count_for_pager def query_for_pager query = has_id? ? 'WHERE id >= ? AND id < ?' : 'LIMIT ?,?' - "SELECT #{columns.map{|c| "`"+c[:name]+"`"}.join(", ")} FROM `#{name}` #{query}" + cols = columns.map do |c| + if "geometry" == c[:type] + "AsWKT(`#{c[:name]}`) as `#{c[:name]}`" + else + "`#{c[:name]}`" + end + end + "SELECT #{cols.join(", ")} FROM `#{name}` #{query}" end end @@ -162,7 +176,7 @@ def initialize(options) @host, @user, @passwd, @db, @port, @sock, @flag = options.mysqlhostname('localhost'), options.mysqlusername, options.mysqlpassword, options.mysqldatabase, - options.mysqlport, options.mysqlsocket + options.mysqlport, options.mysqlsocket(nil) connect end @@ -188,4 +202,4 @@ def paginated_read(table, page_size) end end -end \ No newline at end of file +end diff --git a/lib/mysql2psql/postgres_db_writer.rb b/lib/mysql2psql/postgres_db_writer.rb index f636c9f..5949e3c 100644 --- a/lib/mysql2psql/postgres_db_writer.rb +++ b/lib/mysql2psql/postgres_db_writer.rb @@ -65,7 +65,7 @@ def write_table(table) CACHE 1 EOF - @conn.exec "SELECT pg_catalog.setval('#{table.name}_#{serial_key}_seq', #{maxval}, true)" + @conn.exec sqlfor_set_serial_sequence(table,serial_key,maxval) end if @conn.server_version < 80200 @@ -164,16 +164,25 @@ def write_contents(table, reader) if counter % 5000 == 0 @conn.put_copy_end + res = @conn.get_result + if res.cmdtuples != 5000 + puts "\nWARNING: #{table.name} expected 5000 tuple inserts got #{res.cmdtuples} at row #{counter}\n" + end @conn.exec(copy_line) end end - _time2 = Time.now - puts "\n#{_counter} rows loaded in #{((_time2 - _time1) / 60).round}min #{((_time2 - _time1) % 60).round}s" -# @conn.putline(".\n") @conn.put_copy_end + if _counter && (_counter % 5000) > 0 + res = @conn.get_result + if res.cmdtuples != (_counter % 5000) + puts "\nWARNING: table #{table.name} expected #{_counter % 5000} tuple inserts got #{res.cmdtuples}\n" + end + end + _time2 = Time.now + puts "\n#{table.name} #{_counter} rows loaded in #{((_time2 - _time1) / 60).round}min #{((_time2 - _time1) % 60).round}s" end - + end -end \ No newline at end of file +end diff --git a/lib/mysql2psql/postgres_file_writer.rb b/lib/mysql2psql/postgres_file_writer.rb index 12d39f4..5fc2ad7 100644 --- a/lib/mysql2psql/postgres_file_writer.rb +++ b/lib/mysql2psql/postgres_file_writer.rb @@ -33,7 +33,7 @@ def truncate(table) EOF if serial_key @f << <<-EOF -SELECT pg_catalog.setval(pg_get_serial_sequence('#{table.name}', '#{serial_key}'), #{maxval}, true); +#{sqlfor_reset_serial_sequence(table,serial_key,maxval)} EOF end end @@ -69,8 +69,7 @@ def write_table(table) NO MINVALUE CACHE 1; - -SELECT pg_catalog.setval('#{table.name}_#{serial_key}_seq', #{maxval}, true); +#{sqlfor_set_serial_sequence(table,serial_key,maxval)} EOF end diff --git a/lib/mysql2psql/postgres_writer.rb b/lib/mysql2psql/postgres_writer.rb index 9609ed4..56988c6 100644 --- a/lib/mysql2psql/postgres_writer.rb +++ b/lib/mysql2psql/postgres_writer.rb @@ -16,8 +16,7 @@ def column_type(column) def column_type_info(column) if column[:auto_increment] return "integer DEFAULT nextval('#{column[:table_name]}_#{column[:name]}_seq'::regclass) NOT NULL" - end - + end default = column[:default] ? " DEFAULT #{column[:default] == nil ? 'NULL' : "'"+PGconn.escape(column[:default])+"'"}" : nil null = column[:null] ? "" : " NOT NULL" type = @@ -29,7 +28,6 @@ def column_type_info(column) "character(#{column[:length]})" when "varchar" default = default + "::character varying" if default - # puts "VARCHAR: #{column.inspect}" "character varying(#{column[:length]})" # Integer and numeric types @@ -39,19 +37,19 @@ def column_type_info(column) when "bigint" default = " DEFAULT #{column[:default].nil? ? 'NULL' : column[:default].to_i}" if default "bigint" - when "tinyint" + when /tinyint|smallint/ default = " DEFAULT #{column[:default].nil? ? 'NULL' : column[:default].to_i}" if default "smallint" when "boolean" default = " DEFAULT #{column[:default].to_i == 1 ? 'true' : 'false'}" if default "boolean" - when "float" + when "real" default = " DEFAULT #{column[:default].nil? ? 'NULL' : column[:default].to_f}" if default - "real" - when "float unsigned" + "double precision" + when /float/ default = " DEFAULT #{column[:default].nil? ? 'NULL' : column[:default].to_f}" if default - "real" + "double precision" when "decimal" default = " DEFAULT #{column[:default].nil? ? 'NULL' : column[:default]}" if default "numeric(#{column[:length] || 10}, #{column[:decimals] || 0})" @@ -94,6 +92,8 @@ def column_type_info(column) "text" when "text" "text" + when "geometry" + "geometry" when /^enum/ default = default + "::character varying" if default enum = column[:type].gsub(/enum|\(|\)/, '') @@ -137,7 +137,14 @@ def process_row(table, row) def truncate(table) end - + + def sqlfor_set_serial_sequence(table,serial_key,maxval) + "SELECT pg_catalog.setval('#{table.name}_#{serial_key}_seq', #{maxval}, true);" + end + def sqlfor_reset_serial_sequence(table,serial_key,maxval) + "SELECT pg_catalog.setval(pg_get_serial_sequence('#{table.name}', '#{serial_key}'), #{maxval}, true);" + end + end -end \ No newline at end of file +end diff --git a/lib/mysql2psql/version.rb b/lib/mysql2psql/version.rb index c61d5c1..b791a1a 100644 --- a/lib/mysql2psql/version.rb +++ b/lib/mysql2psql/version.rb @@ -2,7 +2,7 @@ class Mysql2psql module Version MAJOR = 0 MINOR = 1 - PATCH = 0 + PATCH = 1 STRING = [MAJOR, MINOR, PATCH].compact.join('.') end diff --git a/mysql2psql.gemspec b/mysql2psql.gemspec index fb52551..8ad2931 100644 --- a/mysql2psql.gemspec +++ b/mysql2psql.gemspec @@ -1,16 +1,15 @@ # Generated by jeweler # DO NOT EDIT THIS FILE DIRECTLY -# Instead, edit Jeweler::Tasks in Rakefile, and run the gemspec command +# Instead, edit Jeweler::Tasks in Rakefile, and run 'rake gemspec' # -*- encoding: utf-8 -*- Gem::Specification.new do |s| s.name = %q{mysql2psql} - s.version = "0.1.0" + s.version = "0.1.1" s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version= s.authors = ["Max Lapshin ", "Anton Ageev ", "Samuel Tribehou ", "Marco Nenciarini ", "James Nobis ", "quel ", "Holger Amann ", "Maxim Dobriakov ", "Michael Kimsal ", "Jacob Coby ", "Neszt Tibor ", "Miroslav Kratochvil ", "Paul Gallagher "] - s.date = %q{2010-09-19} - s.default_executable = %q{mysql2psql} + s.date = %q{2011-05-02} s.description = %q{It can create postgresql dump from mysql database or directly load data from mysql to postgresql (at about 100 000 records per minute). Translates most data types and indexes.} s.email = %q{gallagher.paul@gmail.com} @@ -19,72 +18,70 @@ Gem::Specification.new do |s| "README.rdoc" ] s.files = [ - ".gitignore", - "MIT-LICENSE", - "README.rdoc", - "Rakefile", - "bin/mysql2psql", - "lib/mysql2psql.rb", - "lib/mysql2psql/config.rb", - "lib/mysql2psql/config_base.rb", - "lib/mysql2psql/converter.rb", - "lib/mysql2psql/errors.rb", - "lib/mysql2psql/mysql_reader.rb", - "lib/mysql2psql/postgres_db_writer.rb", - "lib/mysql2psql/postgres_file_writer.rb", - "lib/mysql2psql/postgres_writer.rb", - "lib/mysql2psql/version.rb", - "lib/mysql2psql/writer.rb", - "mysql2psql.gemspec", - "test/fixtures/config_all_options.yml", - "test/fixtures/seed_integration_tests.sql", - "test/integration/convert_to_db_test.rb", - "test/integration/convert_to_file_test.rb", - "test/integration/converter_test.rb", - "test/integration/mysql_reader_base_test.rb", - "test/integration/mysql_reader_test.rb", - "test/integration/postgres_db_writer_base_test.rb", - "test/lib/ext_test_unit.rb", - "test/lib/test_helper.rb", - "test/units/config_base_test.rb", - "test/units/config_test.rb", - "test/units/postgres_file_writer_test.rb" + "CHANGELOG", + "MIT-LICENSE", + "README.rdoc", + "Rakefile", + "bin/mysql2psql", + "lib/mysql2psql.rb", + "lib/mysql2psql/config.rb", + "lib/mysql2psql/config_base.rb", + "lib/mysql2psql/converter.rb", + "lib/mysql2psql/errors.rb", + "lib/mysql2psql/mysql_reader.rb", + "lib/mysql2psql/postgres_db_writer.rb", + "lib/mysql2psql/postgres_file_writer.rb", + "lib/mysql2psql/postgres_writer.rb", + "lib/mysql2psql/version.rb", + "lib/mysql2psql/writer.rb", + "mysql2psql.gemspec", + "test/fixtures/config_all_options.yml", + "test/fixtures/seed_integration_tests.sql", + "test/integration/convert_to_db_test.rb", + "test/integration/convert_to_file_test.rb", + "test/integration/converter_test.rb", + "test/integration/mysql_reader_base_test.rb", + "test/integration/mysql_reader_test.rb", + "test/integration/postgres_db_writer_base_test.rb", + "test/lib/ext_test_unit.rb", + "test/lib/test_helper.rb", + "test/units/config_base_test.rb", + "test/units/config_test.rb", + "test/units/postgres_file_writer_test.rb" ] s.homepage = %q{http://github.com/tardate/mysql2postgresql} - s.rdoc_options = ["--charset=UTF-8"] s.require_paths = ["lib"] - s.rubygems_version = %q{1.3.7} + s.rubygems_version = %q{1.7.2} s.summary = %q{Tool for converting mysql database to postgresql} s.test_files = [ "test/integration/convert_to_db_test.rb", - "test/integration/convert_to_file_test.rb", - "test/integration/converter_test.rb", - "test/integration/mysql_reader_base_test.rb", - "test/integration/mysql_reader_test.rb", - "test/integration/postgres_db_writer_base_test.rb", - "test/lib/ext_test_unit.rb", - "test/lib/test_helper.rb", - "test/units/config_base_test.rb", - "test/units/config_test.rb", - "test/units/postgres_file_writer_test.rb" + "test/integration/convert_to_file_test.rb", + "test/integration/converter_test.rb", + "test/integration/mysql_reader_base_test.rb", + "test/integration/mysql_reader_test.rb", + "test/integration/postgres_db_writer_base_test.rb", + "test/lib/ext_test_unit.rb", + "test/lib/test_helper.rb", + "test/units/config_base_test.rb", + "test/units/config_test.rb", + "test/units/postgres_file_writer_test.rb" ] if s.respond_to? :specification_version then - current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION s.specification_version = 3 if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then s.add_runtime_dependency(%q, ["= 2.8.1"]) - s.add_runtime_dependency(%q, ["= 0.9.0"]) + s.add_runtime_dependency(%q, ["~> 0.11.0"]) s.add_development_dependency(%q, [">= 2.1.1"]) else s.add_dependency(%q, ["= 2.8.1"]) - s.add_dependency(%q, ["= 0.9.0"]) + s.add_dependency(%q, ["~> 0.11.0"]) s.add_dependency(%q, [">= 2.1.1"]) end else s.add_dependency(%q, ["= 2.8.1"]) - s.add_dependency(%q, ["= 0.9.0"]) + s.add_dependency(%q, ["~> 0.11.0"]) s.add_dependency(%q, [">= 2.1.1"]) end end diff --git a/test/fixtures/config_all_options.yml b/test/fixtures/config_all_options.yml index 06d0b83..cdb1be2 100644 --- a/test/fixtures/config_all_options.yml +++ b/test/fixtures/config_all_options.yml @@ -1,7 +1,7 @@ mysql: hostname: localhost port: 3306 - socket: /tmp/mysql.sock + socket: username: somename password: secretpassword database: somename diff --git a/test/fixtures/seed_integration_tests.sql b/test/fixtures/seed_integration_tests.sql index 8163255..58dc8a7 100644 --- a/test/fixtures/seed_integration_tests.sql +++ b/test/fixtures/seed_integration_tests.sql @@ -4,21 +4,63 @@ DROP TABLE IF EXISTS numeric_types_basics; CREATE TABLE numeric_types_basics ( id int, f_tinyint TINYINT, + f_tinyint_u TINYINT UNSIGNED, f_smallint SMALLINT, + f_smallint_u SMALLINT UNSIGNED, f_mediumint MEDIUMINT, + f_mediumint_u MEDIUMINT UNSIGNED, f_int INT, + f_int_u INT UNSIGNED, f_integer INTEGER, + f_integer_u INTEGER UNSIGNED, f_bigint BIGINT, + f_bigint_u BIGINT UNSIGNED, f_real REAL, f_double DOUBLE, f_float FLOAT, + f_float_u FLOAT UNSIGNED, f_decimal DECIMAL, f_numeric NUMERIC ); INSERT INTO numeric_types_basics VALUES -(1,1,1,1,1,1,1,1,1,1,1,1), -(2,2,2,2,2,2,2,2,2,2,2,2), -(23,23,23,23,23,23,23,23,23,23,23,23); +( 1, 2, 3, 4, 5, 6, 7, 8, 9,10,11,12,13,14,15,16,17,18,19), +( 2, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1), +( 3,23,23,23,23,23,23,23,23,23,23,23,23,23,23,23,23,23,23), +( 4, -128, 0,-32768, 0,-8388608, 0,-2147483648, 0,-2147483648, 0,-9223372036854775808, 0, 1, 1, 1, 1, 1, 1), +( 5, 127, 255, 32767, 65535, 8388607, 16777215, 2147483647, 4294967295, 2147483647, 4294967295, 9223372036854775807, 18446744073709551615, 1, 1, 1, 1, 1, 1); + + +DROP TABLE IF EXISTS basic_autoincrement; +CREATE TABLE basic_autoincrement ( + auto_id INT(11) NOT NULL AUTO_INCREMENT, + auto_dummy INT, + PRIMARY KEY (auto_id) +); + +INSERT INTO basic_autoincrement(auto_dummy) VALUES +(1),(2),(23); + +-- see GH#22 float conversion error +DROP TABLE IF EXISTS numeric_type_floats; +CREATE TABLE numeric_type_floats ( + latitude FLOAT, + longitude FLOAT +); + +INSERT INTO numeric_type_floats(latitude,longitude) VALUES +(1.1,2.2); + +-- see GH#18 smallint error +DROP TABLE IF EXISTS gh18_smallint; +CREATE TABLE gh18_smallint ( + s_smallint SMALLINT, + u_smallint SMALLINT UNSIGNED +); + +INSERT INTO gh18_smallint(s_smallint,u_smallint) VALUES +(-32768,32767), +(-1,0), +(32767,65535); diff --git a/test/integration/convert_to_db_test.rb b/test/integration/convert_to_db_test.rb index 249c45e..47190a0 100644 --- a/test/integration/convert_to_db_test.rb +++ b/test/integration/convert_to_db_test.rb @@ -24,6 +24,8 @@ def teardown def test_table_creation assert_true @@mysql2psql.writer.exists?('numeric_types_basics') + assert_true @@mysql2psql.writer.exists?('basic_autoincrement') + assert_true @@mysql2psql.writer.exists?('numeric_type_floats') end end \ No newline at end of file diff --git a/test/integration/convert_to_file_test.rb b/test/integration/convert_to_file_test.rb index f7c440c..8364701 100644 --- a/test/integration/convert_to_file_test.rb +++ b/test/integration/convert_to_file_test.rb @@ -5,6 +5,12 @@ class ConvertToFileTest < Test::Unit::TestCase class << self + + # This is a suite of tests to verify conversion of full schema and data to file. + # The export is done once in the class setup. + # Tests inspect specific features of the converted file, + # the contents of which are preloaded as the :content attribute of this class + # def startup seed_test_database @@options=get_test_config_by_label(:localmysql_to_file_convert_all) @@ -23,44 +29,83 @@ def teardown def content @@content end - + + # verify table creation def test_table_creation assert_not_nil content.match('DROP TABLE IF EXISTS "numeric_types_basics" CASCADE') assert_not_nil content.match(/CREATE TABLE "numeric_types_basics"/) end + # tests for the conversion of numeric types + def get_basic_numerics_match(column) + Regexp.new('CREATE TABLE "numeric_types_basics".*"' + column + '" ([^\n]*).*\)', Regexp::MULTILINE).match( content ) + end def test_basic_numerics_tinyint - assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_tinyint" smallint,.*\)', Regexp::MULTILINE).match( content ) + match = get_basic_numerics_match( 'f_tinyint' ) + assert_match /smallint/,match[1] + end + def test_basic_numerics_tinyint_u + match = get_basic_numerics_match( 'f_tinyint_u' ) + assert_match /smallint/,match[1] end def test_basic_numerics_smallint - assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_smallint" integer,.*\)', Regexp::MULTILINE).match( content ) + match = get_basic_numerics_match( 'f_smallint' ) + assert_match /smallint/,match[1] + end + def test_basic_numerics_smallint_u + match = get_basic_numerics_match( 'f_smallint_u' ) + assert_match /integer/,match[1] end def test_basic_numerics_mediumint - assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_mediumint" integer,.*\)', Regexp::MULTILINE).match( content ) + match = get_basic_numerics_match( 'f_mediumint' ) + assert_match /integer/,match[1] end def test_basic_numerics_int - assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_int" integer,.*\)', Regexp::MULTILINE).match( content ) + match = get_basic_numerics_match( 'f_int' ) + assert_match /integer/,match[1] end def test_basic_numerics_integer - assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_integer" integer,.*\)', Regexp::MULTILINE).match( content ) + match = get_basic_numerics_match( 'f_integer' ) + assert_match /integer/,match[1] end def test_basic_numerics_bigint - assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_bigint" bigint,.*\)', Regexp::MULTILINE).match( content ) + match = get_basic_numerics_match( 'f_bigint' ) + assert_match /bigint/,match[1] + #assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_bigint" bigint,.*\)', Regexp::MULTILINE).match( content ) end def test_basic_numerics_real - assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_real" double precision,.*\)', Regexp::MULTILINE).match( content ) + match = get_basic_numerics_match( 'f_real' ) + assert_match /double precision/,match[1] + #assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_real" double precision,.*\)', Regexp::MULTILINE).match( content ) end def test_basic_numerics_double - assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_double" double precision,.*\)', Regexp::MULTILINE).match( content ) + match = get_basic_numerics_match( 'f_double' ) + assert_match /double precision/,match[1] + #assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_double" double precision,.*\)', Regexp::MULTILINE).match( content ) end def test_basic_numerics_float - assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_float" numeric\(20, 0\),.*\)', Regexp::MULTILINE).match( content ) + match = get_basic_numerics_match( 'f_float' ) + assert_match /double precision/,match[1] + #assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_float" double precision.*\)', Regexp::MULTILINE).match( content ) + end + def test_basic_numerics_float_u + match = get_basic_numerics_match( 'f_float_u' ) + assert_match /double precision/,match[1] + #assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_float_u" double precision.*\)', Regexp::MULTILINE).match( content ) end def test_basic_numerics_decimal - assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_decimal" numeric\(10, 0\),.*\)', Regexp::MULTILINE).match( content ) + match = get_basic_numerics_match( 'f_decimal' ) + assert_match /numeric/,match[1] + #assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_decimal" numeric\(10, 0\),.*\)', Regexp::MULTILINE).match( content ) end def test_basic_numerics_numeric - assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_numeric" numeric\(10, 0\)[\w\n]*\)', Regexp::MULTILINE).match( content ) + match = get_basic_numerics_match( 'f_numeric' ) + assert_match /numeric/,match[1] + #assert_not_nil Regexp.new('CREATE TABLE "numeric_types_basics".*"f_numeric" numeric\(10, 0\)[\w\n]*\)', Regexp::MULTILINE).match( content ) end + # test autoincrement handling + def test_autoincrement + assert_not_nil Regexp.new('CREATE TABLE "basic_autoincrement".*"auto_id" integer DEFAULT.*\)', Regexp::MULTILINE).match( content ) + end end \ No newline at end of file diff --git a/test/integration/mysql_reader_test.rb b/test/integration/mysql_reader_test.rb index 87c1be3..b6506e5 100644 --- a/test/integration/mysql_reader_test.rb +++ b/test/integration/mysql_reader_test.rb @@ -31,7 +31,7 @@ def test_tables_collection assert_equal 'numeric_types_basics', values[0].name end def test_paginated_read - expected_rows=3 + expected_rows=5 page_size=2 expected_pages=(1.0 * expected_rows / page_size).ceil diff --git a/test/lib/test_helper.rb b/test/lib/test_helper.rb index ea49633..4b77519 100644 --- a/test/lib/test_helper.rb +++ b/test/lib/test_helper.rb @@ -12,7 +12,8 @@ def seed_test_database options=get_test_config_by_label(:localmysql_to_file_convert_nothing) seedfilepath = "#{File.dirname(__FILE__)}/../fixtures/seed_integration_tests.sql" - rc=system("mysql -u#{options.mysqlusername} #{options.mysqldatabase} < #{seedfilepath}") + mysql_cmd = `which mysql`.empty? ? 'mysql5' : 'mysql' + rc=system("#{mysql_cmd} -u#{options.mysqlusername} #{options.mysqldatabase} < #{seedfilepath}") raise StandardError unless rc return true rescue