Commit Graph

8149 Commits

Author SHA1 Message Date
Rhea Parekh fe4cad15a4 widgets: Add todo widget. 2018-07-10 11:18:05 +05:30
Yashashvi Dave 649fccde6b zerver/lib/events.py: Fix custom field values not updated in apply_events.
Custom profile field value are stored in different structure compare to
other profile fields in events, so generic way to update fields wasn't
updating custom profile fields in `apply_event` function.

Fix this by adding check for custom fields in `apply_event`.

This also adds the appropriate test_events test to verify this code path.

Fixes part of #9875.
2018-07-10 10:30:23 +05:30
Aditya Bansal 0c5dadf320 actions.py: Extract topic history list generation as a function.
We extract out the logic for generating a list of all historical
topics for a given stream as a separate function.  This avoids code
duplication when we add the similar code path for grabbing all topics
for web public streams.
2018-07-10 10:22:07 +05:30
Shubham Dhama 49deb5acd3 bugdown: Move global variables to arguments.py.
This has two advantages;

* We can split bugdown/__init__.py into several modules, and each
  module can access these arguments by importing these

* We get rid of the super-ugly `global db_data` construct, replacing
  it with a only slightly ugly monkey-ish patching of the
  `zerver.lib.bugdown.arguments` module, which is at least
  considerably more clear on reading as to what it's purpose is.
2018-07-09 15:45:45 +05:30
Shubham Dhama b9a8c4df8c bugdown: Move BugdownRenderingException to exceptions.py. 2018-07-09 15:45:45 +05:30
Rishi Gupta b5753d0ddc billing: Add initial support for seat based plans.
The main remaining todo for correctly populating
RealmAuditLog.requires_billing_update is supporting the de-seating (and
corresponding re-seating) that happens after being offline for two weeks.
2018-07-09 14:33:08 +05:30
Rishi Gupta 16334a1ba7 billing: Update billing system. 2018-07-09 14:33:07 +05:30
Aditya Bansal 872e8c1d7b test_fixtures: Fix bug with run_db_migrations for test platform.
In this commit we are fixing a kinda serious un-noticed bug with
the way run_db_migrations worked for test db.
Basically run_db_migrations runs new migrations on db (dev or test).
When we talk about the dev platform this process is straight forward.
We have a single DB zulip which was once created and now has some data.
Introduction of new migration causes a schema change or does something
else but bottom line being we just migrate the zulip DB and stuff works
fine.

Now coming to zulip test db (zulip_test) situation is a bit complex
in comparision to dev db. Basically this is because we make use of
what we call zulip_test_template to make test fixture restoration
after tests run fast. Now before we introduced the performance
optimisation of just doing migrations when possible, introduction of
a migration would ideally result in provisioning do a full rebuild of
the test database. When that used to happen sequence of events used to
be something like this:
* Create a zulip_test db from zulip_test_base template (An absolute
basic schema holding)
* Migrate and populate the zulip_test db.
* Create/Re-create zulip_test_template from the latest zulip_test.

Now after we introduced just do migrations instead of full db rebuild
when possible, what used to happen was that zulip_test db got
successfully migrated but when test suites would run they would try to
create zulip_test from zulip_test_template (so that individual tests
don't affect each other on db level).
This is where the problem resides; zulip_test_template wasn't migrated
and we just scrapped zulip_test and re-created it using
zulip_test_template as a template and hence zulip_test will not hold the
latest schema.

This is what we fix in this commit.
2018-07-09 14:17:53 +05:30
Armaan Ahluwalia 54d3d8e8b3 webpack: Transition app.js to be compiled by webpack.
This commit moves all files previously under the 'app' bundle in
the Django pipeline to being compiled by webpack under the 'app'
entry point. In the process, it moves assets under the app entry
to a file called app.js that consumes all relevant css and js files.

This commit also edits the webpack config to be able to expose certain
variables for third party libraries that are currently required by
some modules. This is bad coding form and should be refactored to
requiring whatever dependencies a module may have; we're just
deferring that to the future to simplify the series of transitions we
need to do here. The variable exposure is done using expose-loader in
webpack.

The app/index.html template is edited to override the newly introduced
'commonjs' block in the base template. This is done as a temporary
measure so as not to disrupt other pages on the app during the transition.

It also fixes the value of the 'this' context that was being inferred
as window by third party libraries. This is done using imports-loader
in the webpack config.  This is also messy and probably isn't how we
want things to work long term.
2018-07-05 11:03:08 +02:00
Raymond Akornor 5ef6689a69 minified_js: Clean up unused imports. 2018-07-04 17:38:55 +02:00
Yago González fa2daf0044 api docs: Migrate POST /users/me/subscriptions to OpenAPI. 2018-07-03 20:15:34 +02:00
Yago González f3277500de api docs: Add missing space in exception's message. 2018-07-03 20:11:05 +02:00
Yago González 2d1adc8d99 api docs: Allow validation against schemas for any response. 2018-07-03 20:11:05 +02:00
Tim Abbott 5a99118b3e auth: Restore a minimal SocialAuthMixin.
We need to do a small monkey-patching of python-social-auth to ensure
that it doesn't 500 the request when a user does something funny in
their browser (e.g. using the back button in the auth flow) that is
fundamentally a user error, not a server error.

This was present in the pre-rewrite version of our Social auth
codebase, without clear documentation; I've fixed the explanation
part here.

It's perhaps worth investigating with the core social auth team
whether there's a better way to do this.
2018-07-03 18:53:59 +02:00
Tim Abbott c9b0c0add4 github: Refactor email extraction to use the full emails data set.
It's possible to make GitHub social authentication support letting the
user pick which of their verified email addresses to pick, using the
python-social-auth pipeline feature.  We need to add an additional
screen to let the user pick, so we're not adding support for that now,
but this at least migrates this to use the data set of all emails that
have been verified as associated with the user's GitHub account (and
we just assume the user wants their primary email).

This also fixes the inability for very old GitHub accounts (where the
`email` field in the details might be a string the user wanted on
their GitHub profile page) to using GitHub auth to login.

Fixes #9127.
2018-07-03 18:35:29 +02:00
Anders Kaseorg d8ba378050 test_slack_importer: Remove backslashes wrongly copied from JSON data
https://github.com/houstondatavis/slack-export/blob/master/users.json

JSON or JavaScript decodes "\/" to / (and some encoders always write
"\/" to avoid accidentally creating a </script> tag), while Python
assumes "\/" is a typo for "\\/" and decodes it to \/.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-03 16:54:46 +02:00
Anders Kaseorg 037f696d26 Enable pycodestyle W605 (invalid escape sequence).
The only changes visible at the AST level, checked using
https://github.com/asottile/astpretty, are

zerver/lib/test_fixtures.py:
'\x1b\\[(1|0)m' ↦ '\\x1b\\[(1|0)m'
'\\[[X| ]\\] (\\d+_.+)\n' ↦ '\\[[X| ]\\] (\\d+_.+)\\n'

which is fine because re treats '\\x1b' and '\\n' the same way as
'\x1b' and '\n'.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-03 16:54:46 +02:00
Rhea Parekh fdef1d8d91 zcommand: Raise error when command doesn't begin with a slash. 2018-07-01 20:32:36 -04:00
Rhea Parekh b22d266667 widgets: Add question for poll widget in the message itself.
Use the command '/poll question?', to start a question.
2018-07-01 19:56:19 -04:00
Lenny Jagielski b9393387e1 integrations: Fix heroku integration logging.
Now, every line in the commit log is properly quoted, but using a
Zulip markdown block quote (rather than just a single `>`).

Fixes: #9792
2018-07-01 12:43:31 -07:00
Eeshan Garg bf175f6331 webhooks/github: Add support for PR review requests.
Fixes: #9732.
2018-07-01 12:40:45 -07:00
Eeshan Garg 04ed123214 webhooks/github: Be more explicit about unsupported PR events.
This was technically a bug. For events that aren't unsupported
intentionally, the control should fall to the line that raises
UnknownWebhookEventType, and shouldn't be handled by anything else.

The events that are intentionally unsupported should be handled
more explicitly.
2018-07-01 12:40:45 -07:00
Yago González d151f4a234 messages: Allow fetching unedited messages' history.
When GETting an unedited message's edit history, the server wasn't able
to reply properly and produced a 500 error.

Now when that happens, we return a message history that only contains
the original message.
2018-07-01 08:16:59 -07:00
Rhea Parekh 838ab2fce5 export: Add variable MESSAGE_BATCH_CHUNK_SIZE in export.py.
Also use this variable in slack_data_to_zulip_data.
2018-07-01 07:08:13 -07:00
Rhea Parekh 6b7b6b38ad slack import: Write messages batch-wise.
Messages can be bulky, and storing them in a single
data structure can cause a memory error.

In this commit, the messages are written to a file
batch-wise, thus avoiding the memory error.
2018-07-01 07:08:13 -07:00
Rhea Parekh 7f6c174099 slack import: Add 'id_list' field in channel_message_to_zerver_message.
The id_list would help to store the associated max ID
state between subsequent calls, which will help
in batch-wise processing of the messages.
2018-07-01 07:08:13 -07:00
Rhea Parekh af20ef4789 slack import: Save messages within convert_slack_workspace_messages.
Previously, the messages where being stored in a output file from
outside the function 'convert_slack_workspace_messages', but
now we store it from the inside the mentioned function.
This will help in processing and saving the messages batch-wise
so as to avoid a memory error.

Reactions are returned separately from 'convert_slack_workspace_messages'
rather than 'message_json'.

Also updated test for 'convert_slack_workspace_messages' and an additional
test for reactions is added.
2018-07-01 07:08:13 -07:00
Rhea Parekh 2f88ca7446 slack import: Import skype and phone data of users. 2018-07-01 07:05:40 -07:00
Eeshan Garg 061e760d1e webhooks: Add Clubhouse integration. 2018-07-01 04:01:17 -07:00
Tim Abbott 5f7691b74e upload: Remove unnecessary use of has_request_variables.
All the parameters for this function are parsed in urls.py.
2018-07-01 01:47:03 -07:00
Eeshan Garg e1e4d523aa Replace remaining user-facing occurences of the term "invite-only".
Fixes #9611.
2018-06-27 12:34:29 -07:00
Steve Howell a2a695dfa7 test flake: Prevent flaky test with less common id.
This fixes a test flake introduced here:

    317a2fff2a

We need a higher bogus bot owner id to prevent
flakes where our userid sequence gets to 100.  (Tests
aren't completely deterministic in what data you
use, since sequences don't get rolled back when
you roll back transactions.)
2018-06-26 16:56:24 -04:00
Rishi Gupta 2de9029a69 stream settings: Add tests for weekly stream traffic calculation. 2018-06-23 12:48:15 -07:00
Rishi Gupta dc7cfd3eb7 stream settings: Fix stream_age calculation in weekly traffic calculation. 2018-06-23 12:48:15 -07:00
Rishi Gupta 3054b175c9 stream setting: Fix mypy error in get_average_weekly_stream_traffic.
Not sure why mypy wasn't catching this.
2018-06-23 12:48:15 -07:00
Rishi Gupta 15b236a21f stream settings: Show non-zero stream traffic if traffic is greater than 0.
An estimated traffic of 0 suggests a stream is dead, and has pretty
different semantics from any non-zero value. So we should round up any
number between 0 and 1 to 1.
2018-06-23 12:48:15 -07:00
Rishi Gupta 5eccabc3c5 stream settings: Set average_weekly_traffic to -1 if stream is new.
We don't ever use this value, but it's confusing to have the incorrect
calculation in the code.

Ideally we would set this to "None", but I don't know the code well enough
to be confident nothing would break.
2018-06-23 12:48:15 -07:00
Rishi Gupta f63bcb99d7 stream settings: Refactor out STREAM_TRAFFIC_CALCULATION_MIN_AGE_DAYS. 2018-06-23 12:48:15 -07:00
Rishi Gupta ccf5f1ac89 stream settings: Fix weekly stream traffic computation.
Previously, if stream_age were 13, this number would be off by almost a
factor of 2.
2018-06-23 12:48:15 -07:00
Yashashvi Dave 317a2fff2a bots settings: Improve API for editing bot owners to refer to users by ID.
Fixes #9502
2018-06-23 12:47:50 -07:00
Vishnu Ks 6e6e2e03da management: Use tokenized no-reply address in send_test_email. 2018-06-23 12:03:30 -07:00
Vishnu Ks 1b179ca530 signup: Prevent users from signing up with email containing +. 2018-06-23 12:03:30 -07:00
Vishnu Ks a88e44e254 registration: Use tokenized noreply address in email change. 2018-06-23 12:03:30 -07:00
Vishnu Ks e34fcf982f registration: Use tokenized noreply address in user invite. 2018-06-23 12:03:30 -07:00
Vishnu Ks 994e1a2154 registration: Use tokenized noreply address in password reset. 2018-06-23 12:03:30 -07:00
Vishnu Ks 2b1424e51a registration: Use tokenized noreply address in signup confirmation. 2018-06-23 12:03:30 -07:00
Vishnu Ks 74351fb250 send_email: Add function to generate tokenized noreply in FromAddress. 2018-06-23 12:03:30 -07:00
Yago González 51f107f5bc api docs: Fix indentation. 2018-06-23 12:42:11 +02:00
Yago González 8b420da653 api docs: Remove unnecessary imports. 2018-06-23 12:42:11 +02:00
Yago González 8e50564717 api docs: Document the /users/me/<stream_id>/topics endpoint. 2018-06-23 12:42:11 +02:00