Commit Graph

3469 Commits

Author SHA1 Message Date
Shubham Dhama 49deb5acd3 bugdown: Move global variables to arguments.py.
This has two advantages;

* We can split bugdown/__init__.py into several modules, and each
  module can access these arguments by importing these

* We get rid of the super-ugly `global db_data` construct, replacing
  it with a only slightly ugly monkey-ish patching of the
  `zerver.lib.bugdown.arguments` module, which is at least
  considerably more clear on reading as to what it's purpose is.
2018-07-09 15:45:45 +05:30
Shubham Dhama b9a8c4df8c bugdown: Move BugdownRenderingException to exceptions.py. 2018-07-09 15:45:45 +05:30
Rishi Gupta b5753d0ddc billing: Add initial support for seat based plans.
The main remaining todo for correctly populating
RealmAuditLog.requires_billing_update is supporting the de-seating (and
corresponding re-seating) that happens after being offline for two weeks.
2018-07-09 14:33:08 +05:30
Aditya Bansal 872e8c1d7b test_fixtures: Fix bug with run_db_migrations for test platform.
In this commit we are fixing a kinda serious un-noticed bug with
the way run_db_migrations worked for test db.
Basically run_db_migrations runs new migrations on db (dev or test).
When we talk about the dev platform this process is straight forward.
We have a single DB zulip which was once created and now has some data.
Introduction of new migration causes a schema change or does something
else but bottom line being we just migrate the zulip DB and stuff works
fine.

Now coming to zulip test db (zulip_test) situation is a bit complex
in comparision to dev db. Basically this is because we make use of
what we call zulip_test_template to make test fixture restoration
after tests run fast. Now before we introduced the performance
optimisation of just doing migrations when possible, introduction of
a migration would ideally result in provisioning do a full rebuild of
the test database. When that used to happen sequence of events used to
be something like this:
* Create a zulip_test db from zulip_test_base template (An absolute
basic schema holding)
* Migrate and populate the zulip_test db.
* Create/Re-create zulip_test_template from the latest zulip_test.

Now after we introduced just do migrations instead of full db rebuild
when possible, what used to happen was that zulip_test db got
successfully migrated but when test suites would run they would try to
create zulip_test from zulip_test_template (so that individual tests
don't affect each other on db level).
This is where the problem resides; zulip_test_template wasn't migrated
and we just scrapped zulip_test and re-created it using
zulip_test_template as a template and hence zulip_test will not hold the
latest schema.

This is what we fix in this commit.
2018-07-09 14:17:53 +05:30
Yago González fa2daf0044 api docs: Migrate POST /users/me/subscriptions to OpenAPI. 2018-07-03 20:15:34 +02:00
Yago González f3277500de api docs: Add missing space in exception's message. 2018-07-03 20:11:05 +02:00
Yago González 2d1adc8d99 api docs: Allow validation against schemas for any response. 2018-07-03 20:11:05 +02:00
Anders Kaseorg 037f696d26 Enable pycodestyle W605 (invalid escape sequence).
The only changes visible at the AST level, checked using
https://github.com/asottile/astpretty, are

zerver/lib/test_fixtures.py:
'\x1b\\[(1|0)m' ↦ '\\x1b\\[(1|0)m'
'\\[[X| ]\\] (\\d+_.+)\n' ↦ '\\[[X| ]\\] (\\d+_.+)\\n'

which is fine because re treats '\\x1b' and '\\n' the same way as
'\x1b' and '\n'.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-03 16:54:46 +02:00
Rhea Parekh fdef1d8d91 zcommand: Raise error when command doesn't begin with a slash. 2018-07-01 20:32:36 -04:00
Rhea Parekh b22d266667 widgets: Add question for poll widget in the message itself.
Use the command '/poll question?', to start a question.
2018-07-01 19:56:19 -04:00
Rhea Parekh 838ab2fce5 export: Add variable MESSAGE_BATCH_CHUNK_SIZE in export.py.
Also use this variable in slack_data_to_zulip_data.
2018-07-01 07:08:13 -07:00
Rhea Parekh 6b7b6b38ad slack import: Write messages batch-wise.
Messages can be bulky, and storing them in a single
data structure can cause a memory error.

In this commit, the messages are written to a file
batch-wise, thus avoiding the memory error.
2018-07-01 07:08:13 -07:00
Rhea Parekh 7f6c174099 slack import: Add 'id_list' field in channel_message_to_zerver_message.
The id_list would help to store the associated max ID
state between subsequent calls, which will help
in batch-wise processing of the messages.
2018-07-01 07:08:13 -07:00
Rhea Parekh af20ef4789 slack import: Save messages within convert_slack_workspace_messages.
Previously, the messages where being stored in a output file from
outside the function 'convert_slack_workspace_messages', but
now we store it from the inside the mentioned function.
This will help in processing and saving the messages batch-wise
so as to avoid a memory error.

Reactions are returned separately from 'convert_slack_workspace_messages'
rather than 'message_json'.

Also updated test for 'convert_slack_workspace_messages' and an additional
test for reactions is added.
2018-07-01 07:08:13 -07:00
Rhea Parekh 2f88ca7446 slack import: Import skype and phone data of users. 2018-07-01 07:05:40 -07:00
Eeshan Garg 061e760d1e webhooks: Add Clubhouse integration. 2018-07-01 04:01:17 -07:00
Eeshan Garg e1e4d523aa Replace remaining user-facing occurences of the term "invite-only".
Fixes #9611.
2018-06-27 12:34:29 -07:00
Rishi Gupta dc7cfd3eb7 stream settings: Fix stream_age calculation in weekly traffic calculation. 2018-06-23 12:48:15 -07:00
Rishi Gupta 3054b175c9 stream setting: Fix mypy error in get_average_weekly_stream_traffic.
Not sure why mypy wasn't catching this.
2018-06-23 12:48:15 -07:00
Rishi Gupta 15b236a21f stream settings: Show non-zero stream traffic if traffic is greater than 0.
An estimated traffic of 0 suggests a stream is dead, and has pretty
different semantics from any non-zero value. So we should round up any
number between 0 and 1 to 1.
2018-06-23 12:48:15 -07:00
Rishi Gupta 5eccabc3c5 stream settings: Set average_weekly_traffic to -1 if stream is new.
We don't ever use this value, but it's confusing to have the incorrect
calculation in the code.

Ideally we would set this to "None", but I don't know the code well enough
to be confident nothing would break.
2018-06-23 12:48:15 -07:00
Rishi Gupta f63bcb99d7 stream settings: Refactor out STREAM_TRAFFIC_CALCULATION_MIN_AGE_DAYS. 2018-06-23 12:48:15 -07:00
Rishi Gupta ccf5f1ac89 stream settings: Fix weekly stream traffic computation.
Previously, if stream_age were 13, this number would be off by almost a
factor of 2.
2018-06-23 12:48:15 -07:00
Yashashvi Dave 317a2fff2a bots settings: Improve API for editing bot owners to refer to users by ID.
Fixes #9502
2018-06-23 12:47:50 -07:00
Vishnu Ks 1b179ca530 signup: Prevent users from signing up with email containing +. 2018-06-23 12:03:30 -07:00
Vishnu Ks a88e44e254 registration: Use tokenized noreply address in email change. 2018-06-23 12:03:30 -07:00
Vishnu Ks e34fcf982f registration: Use tokenized noreply address in user invite. 2018-06-23 12:03:30 -07:00
Vishnu Ks 74351fb250 send_email: Add function to generate tokenized noreply in FromAddress. 2018-06-23 12:03:30 -07:00
Yago González 8b420da653 api docs: Remove unnecessary imports. 2018-06-23 12:42:11 +02:00
Yago González 8e50564717 api docs: Document the /users/me/<stream_id>/topics endpoint. 2018-06-23 12:42:11 +02:00
Rhea Parekh d066c11b47 zcommand: Strip the slash in the backend.
The slash in command is stripped in the backend,
rather than in the client to make the client code
cleaner.
This would make client code cleaner in the slash
commands which include parameters.
2018-06-20 16:49:29 -04:00
Marco Burstein 02489e5fb8 emoji: Fix a bug where emoticon translation is not done after a newline.
This bug is caused by the conversion of newlines to `<br>` statements,
since `>` is not allowed as a character around an emoticon during
translation.

Also, add a new test case for preventing this bug from occurring in the
future.

Fix #9763.
2018-06-20 08:09:18 -07:00
Vishnu Ks 403f254557 signup: Create get_accounts_for_email function. 2018-06-19 11:25:23 -07:00
neiljp (Neil Pilgrim) e4821875f7 mypy: Improve typing of oembed data, to Dict[str, Any]. 2018-06-19 10:48:38 -07:00
Rhea Parekh d1ba6bae03 import: 'processing_emojis' and 'processing_avatars' should now be True together.
Raise an exception when the fields
'processing_emojis' and 'processing_avatars' are
True at the same time. Also add test for the same.
2018-06-18 23:06:09 +05:30
Rhea Parekh a2a74d9271 export: The records.json IDs should be integer.
In records the IDs like the realm_id and user_profile_id
of 'records.json' should be integers. This was missing in the
S3 backend and this commit fixes that.

Added tests for this as well.
2018-06-18 23:06:09 +05:30
Rhea Parekh 4d21f7f747 import: 'attachment_path' should be saved with the 's3_path' of the record.
For the S3 backend uploads, 'attachment_path' should be
saved with the 's3_path' of the record, as the original
'path' is changed while exporting files from s3. (See
function 'export_files_from_s3' in export.py for reference.)
2018-06-18 23:06:01 +05:30
Rhea Parekh 0730087111 import: Add elif condition for 'processing_emojis' in 'import_uploads_s3'.
'processing_emojis' should have an 'elif' condition here as
we want the function to work for avatars, emojis or uploads
one at a time.
2018-06-18 23:04:18 +05:30
Rhea Parekh f2b5f5a8f9 import: Fix processing_avatars bug in 'import_uploads_s3'.
All the avatars should be processed later on to run the
'ensure_medium_avatar_image' function. This is similar to
'import_uploads_local'.
2018-06-18 22:37:34 +05:30
Rhea Parekh f66ca9a5c3 import: Pass 'processing_emojis' in 'import_uploads_s3'.
'import_uploads_s3' should be passed with the parameter
'processing_emojis' from 'import_uploads'.
2018-06-18 22:35:36 +05:30
Rhea Parekh b2e971b9b1 tests: Add tests for the export file's records.
For the emojis, In 'records.json', the record should contain
the attribute 'file_name', which was missing in the S3 backend.
This commit adds this attribute, as well as tests for the
records of uploads, avatars and emojis in both local and S3 backend.
2018-06-18 09:19:24 -07:00
Yashashvi Dave 8909cb1d15 custom fields: Allow list of users in user type of custom fields.
Allow user to add more than one user-value in user type of custom
fields.

Tweaked by tabbott to improve the models.py code and type annotations.
2018-06-16 09:37:49 -07:00
Eeshan Garg 943a77105b integrations/irc: Capitalize the integration name properly.
It is IRC, not Irc, which is what the default code in Integration
renders.
2018-06-16 08:52:49 -07:00
Vishnu Ks 4da61e2a17 signup: Copy enter_sends value when importing settings. 2018-06-16 08:50:49 -07:00
Vishnu Ks b639778d0f signup: Copy hotspots when importing settings. 2018-06-16 08:50:49 -07:00
Rhea Parekh d0bc8d0736 zcommands: Add zcommand module and separate test module.
Move the zcommands from '/views/messages.py' to
'/lib/zcommand'.

Also, move the zcommand tests from '/tests/test_messages.py'
to '/tests/test_zcommand'.
2018-06-16 10:32:42 -04:00
Shubham Dhama 596ce5b60f invite: Make inviting new users inaccessible for guest users. 2018-06-16 06:33:13 -07:00
Neil Pilgrim ba55d22fdb mypy: Improve MessageOutput typing in export.py.
See the comments above for why this is the correct list of options.
2018-06-14 15:22:56 -07:00
Yashashvi Dave f7f039e772 custom fields: Add endpoint to delete value of custom profile data.
Add delete endpoint at `users/me/profile_data` to delete value of
custom profile data.

Fixes #9669
2018-06-07 08:35:04 -07:00
Adam Birds 5855fe3e53 integrations: Add Zabbix webhook.
With a few changes to the documentations by Eeshan Garg.
2018-06-07 08:31:24 -07:00