Commit Graph

258 Commits

Author SHA1 Message Date
Greg Price 613d093d7d push notifs: Implement APNs with new API.
And it works!

A couple of things still to do:

 * When a device token is no longer active, we'll get HTTP status 410.
   We should then remove the token from the database so we don't keep
   trying to push to it.  This is fairly urgent.

 * The library we're using has a nice asynchronous API, but this
   version doesn't use it.  This is OK now, but async will be
   essential at scale.
2017-08-26 14:16:05 -07:00
Rishi Gupta b32fe9f2a0 features: Update screenshot and add tool to generate.
This adds a new management command that should make generating or
updating screenshots a bit easier in the future.
2017-08-22 11:32:22 -07:00
Tim Abbott d7f8781b06 populate_db: Remove unnecessary print statement. 2017-08-15 12:55:56 -07:00
Tim Abbott 2858c13513 populate_db: Use caching helpers to get users. 2017-08-15 12:55:00 -07:00
Tim Abbott 27b79e7189 populate_db: Fix duplicate test data when run multithreaded.
The previous implementation resulted in each of the 10 threads using
the same random seed and thus same messages.

Fixes #6028.
2017-08-05 11:22:11 -07:00
Darren Fix 69767a43cb Create complex sample messages for dev VM.
Create a generator script to pull lines from a play, enhancing
random lines with emoji, Markdown and other flair.

With numerous contributions from Rein Zustand and Tim Abbott to finish
the project.

Fixes: #1666.
2017-08-01 20:54:39 -07:00
Aditya Bansal 9e372c465c populate_db: Set pointer as message_id of last user sent message. 2017-07-28 11:57:52 -07:00
Harshit Bansal eeb25b6b4b populate_db: Add a test realm emoji while populating databases.
Add `green_tick` realm emoji which can be used while performing tests.
2017-07-27 18:57:06 +00:00
Vishnu Ks 7eaec543f1 zilencer: Use get_user in populate_db.py. 2017-07-18 17:14:05 -07:00
Vishnu Ks df664db7b4 zilencer: Remove unused imports from populate_db.py. 2017-07-18 17:14:05 -07:00
Vishnu Ks 9a44794d14 zilencer: Eliminate get_user_profile_by_email from sync_api_key. 2017-07-18 17:14:05 -07:00
Vishnu Ks b0b809b195 zilencer: Eliminate get_user_profile_by_email from print_initial_password. 2017-07-18 17:14:05 -07:00
Vishnu Ks 754db8ec9f zilencer: Eliminate get_user_profile_by_email from profile_request. 2017-07-18 17:14:05 -07:00
Aditya Bansal f2d6194ae1 actions: Start logging subscription activities in RealmAuditLog. 2017-07-17 17:23:41 -07:00
Aditya Bansal 0c4165a5bd pep8: Add compliance with rule E261 to populate_db.py. 2017-07-11 11:56:50 -07:00
Vaida d5517bae36 Delete the old zulip.com "referrals" system.
This system hasn't been in active use for several years, and had some 
problems with it's design.  So it makes sense to just remove it to declutter
the codebase.

Fixes #5655.
2017-07-07 14:59:18 -07:00
Tim Abbott d6e0960ca2 push_notifications: Fix mypy annotation logic around push tokens.
I'm not 100% confident this is long-term correct, but at least it's
consistent.
2017-07-07 10:54:37 -07:00
Tim Abbott 0606ba88df push_notifications: Verify that token types are valid.
We only have two types of push notification tokens, so we should
validate that in the bouncer code path.
2017-07-07 10:28:36 -07:00
Tim Abbott 40425044c4 push_notifications: Validate APNS token format in bouncer.
This prevents a buggy old Zulip server from forwarding invalid-format
push notification tokens to the push notification bouncer service.

As part of this change, we switch the token from Text to str to match
the rest of the code path.
2017-07-07 10:28:36 -07:00
Rishi Gupta 36cd122905 models: Change default org_type to CORPORATE.
Once we implement org_type-specific features, it'll be easy to change a
corporate realm to a community realm, but hard to go the other way. The main
difference (the main thing that makes migrating from a community realm to a
corporate realm hard) is that you'd have to make everyone sign another terms
of service.
2017-06-29 15:14:58 -07:00
Umair Khan 323fbdefa8 zilencer: Add on_delete in foreign keys.
on_delete will be a required arg for ForeignKey in Django 2.0. Set it
to models.CASCADE on models and in existing migrations if you want to
maintain the current default behavior.
See https://docs.djangoproject.com/en/1.11/ref/models/fields/#django.db.models.ForeignKey.on_delete
2017-06-13 15:14:00 -07:00
Aditya Bansal bcc247a5b1 pep8: Add compliance with rule E261 zilencer/urls.py. 2017-05-31 17:07:15 -07:00
vaibhav 9cf9837f12 webhooks: Add outgoing webhook bot user to development database. 2017-05-29 16:01:23 -07:00
Christian Hudon c80e6edb4e mypy: Declare models with null=True Optional. 2017-05-23 14:36:40 -07:00
Tim Abbott 0b2388bda9 push_notifications: Remove DeviceTokenType logic.
The syntax wasn't valid on Python 3.5, and the new code is somewhat
more readable anyway.
2017-05-16 12:26:55 -07:00
Umair Khan c4e5b75ead push_notification: Push data from Zilencer. 2017-05-16 12:26:55 -07:00
Umair Khan 4a864c7515 push_notification: Send data to notification bouncer. 2017-05-16 12:26:55 -07:00
Tim Abbott c7d2caab6e zilencer: Add auto_now to RemoteZulipServer.last_updated.
This should simplify the logic needed to create one of these.
2017-05-15 17:04:33 -07:00
Umair Khan e649c05aed views.py: Create validator for remote server. 2017-05-11 12:04:16 -07:00
Aditya Bansal bbccd8ab28 pep8: Add compliance with rule E261 to zilencer/models.py. 2017-05-07 23:21:50 -07:00
Aditya Bansal b07273c2d0 pep8: Add compliance with rule E261 to commands/profile_request.py. 2017-05-07 23:21:50 -07:00
Tim Abbott 60225012e6 unread: Add tool for marking all messages as unread for testing.
This tool can save a lot of manual work in testing our unread counts
logic.
2017-04-25 15:40:12 -07:00
Tim Abbott cddee49e75 Add support infrastructure for push notification bouncer service.
This is an incomplete cleaned-up continuation of Lisa Neigut's push
notification bouncer work.  It supports registration and
deregistration of individual push tokens with a central push
notification bouncer server.

It still is missing a few things before we can complete this effort:
* A registration form for server admins to configure their server for
  this service, with tests.
* Code (and tests) for actually bouncing the notifications.
2017-04-18 23:03:06 -07:00
Tim Abbott 8ae052a9d8 populate_db: Set a description for the default realm. 2017-04-17 21:59:21 -07:00
hackerkid b2504084ab Replace timezone.now with timezone_now. 2017-04-16 12:28:56 -07:00
Harshit Bansal ac2172e233 models: Rename RealmAlias model to RealmDomain.
Includes a migration.
2017-04-04 15:48:03 -07:00
Rishi Gupta 8617c2d594 populate_db: Make Iago staff.
Will make it easier to test the /activity page.
2017-03-27 16:43:35 -07:00
Rishi Gupta 30024d0a8f models: Remove Realm.domain. 2017-03-25 19:55:48 -07:00
Tim Abbott 3617ebfd41 api: Rename get_old_messages to get_messages in the backend.
Fixes #1315.
2017-03-23 23:52:44 -07:00
Rishi Gupta ba228b75b1 zilencer: Use Realm.string_id for get_deployment_from_domain.
This is dead code. Getting rid of realm__domain since we're about to remove
that field from Realm.
2017-03-14 17:17:42 -07:00
Rishi Gupta 871c754369 bulk_create: Remove unused function bulk_create_realms. 2017-03-13 14:42:55 -07:00
Tim Abbott 950e06cf32 populate_db: Remove unused Deployment import. 2017-03-06 00:11:53 -08:00
Tim Abbott 10a52147fc zilencer: Remove obsolete create_deployment command. 2017-03-06 00:10:03 -08:00
Tim Abbott e1c1f96f9e zilencer: Eliminate submit_feedback indirection. 2017-03-06 00:01:57 -08:00
Tim Abbott ed5b76f566 zilencer: Move feedback code to zerver/lib/feedback. 2017-03-06 00:01:57 -08:00
Tim Abbott 1f9d93bc96 submit_feedback: remove unused domainish variable. 2017-03-06 00:01:57 -08:00
Tim Abbott 69061e6db4 zilencer: Remove obsolete lookup_endpoints_for_user endpoint.
This endpoint was part of the same obsolete system as
desktop_sso_dispatch.
2017-03-06 00:01:56 -08:00
Tim Abbott 9e8023843a zilencer: Remove obsolete desktop_sso_dispatch.
This feature has been obsolete since when Zulip was released as open
source software, since it's purpose was to avoid putting a "server
url" prompt in the desktop app, and now that prompt is required
anyway.
2017-03-06 00:01:47 -08:00
Raghav Jajodia a3a03bd6a5 mypy: Added Dict, List and Set imports.
Fixed mypy errors associated with the upgrade.
2017-03-04 14:33:44 -08:00
Rishi Gupta 1453a5bfda Change string_id of test zephyr realm from mit to zephyr.
Also changes Realm.is_zephyr_mirror_realm to use string_id=zephyr instead of
domain=mit.edu.
Part of a larger migration away from Realm.domain.
2017-03-04 12:18:01 -08:00
Rishi Gupta 15d60fa7ed Change now() to timezone.now() throughout codebase.
Change `from django.utils.timezone import now` to
`from django.utils import timezone`.

This is both because now() is ambiguous (could be datetime.datetime.now),
and more importantly to make it easier to write a lint rule against
datetime.datetime.now().
2017-02-28 16:03:37 -08:00
Tim Abbott 8691f1466a populate_db: Disable use of memcached when populating database.
This fixes an issue where one would get errors of the form:

`ValueError: unsupported pickle protocol: 3`

in a `run-dev.py` server run against Python 2 if you ran `provision`.
Provision currently runs `populate_db` with Python 3, storing Python 3
based data in memcached, which then can't be read by Python 2.
2017-02-26 21:08:47 -08:00
Tim Abbott 08a7e0db44 render_messages: Fix buggy handling of edit history. 2017-02-18 22:33:26 -08:00
Tim Abbott 080182ad17 populate_db: Fix broken initial presence data.
Apparently, the initial data just set everyone to "API" in a broken way.
2017-02-10 23:52:32 -08:00
Tim Abbott 7e8d3d9a46 populate_db: Add --extra-bots option to create extra bots. 2017-02-10 17:40:45 -08:00
Steve Howell 4be2d6577d Add a new community realm to our test databases.
The realm with string_id of "simple" just has three users
named alice, bob, and cindy for now.  It is useful for testing
scenarios where realms don't have special zulip.com exception
handling.
2017-01-26 19:25:34 -08:00
Tim Abbott dd9e0b8463 errors: Move do_report_error into zerver/lib/. 2017-01-23 23:18:28 -08:00
Tim Abbott e6fc4ae27d errors: Move zilencer/error_notify.py to zerver/lib/. 2017-01-23 23:17:09 -08:00
Tim Abbott 4e171ce787 lint: Clean up E126 PEP-8 rule. 2017-01-23 22:06:13 -08:00
Tim Abbott d6e38e2a5c lint: Clean up E123 PEP-8 rule. 2017-01-23 21:34:26 -08:00
Tim Abbott e9158dd520 lint: Clean up E121 PEP-8 rule. 2017-01-23 21:02:39 -08:00
Tim Abbott c0c9dfb66d populate_db: Fix crash by removing do_send_message.
This old helper has for years been used only by populate_db, and got
buggy (as of a recent refactoring).  So we just call do_send_messages
directly instead.

Fixes the provisioning error we currently get in Travis CI.
2017-01-21 22:07:36 -08:00
Tim Abbott 99c5563bc6 internal_send_message: Make realm argument mandatory.
A lot of care has been taken to ensure we're using the realm that the
message is being sent into, not the realm of the sender, to correctly
handle the logic for cross-realm bot users such as the notifications
bot.
2017-01-21 21:37:30 -08:00
Tommy Ip 3823376b3f zilencer: Fix bare except clause. 2017-01-08 16:22:21 -08:00
Rishi Gupta 2b0a7fd0ba Rename models.get_realm_by_string_id to get_realm.
Finishes the refactoring started in c1bbd8d. The goal of the refactoring is
to change the argument to get_realm from a Realm.domain to a
Realm.string_id. The steps were

* Add a new function, get_realm_by_string_id.

* Change all calls to get_realm to use get_realm_by_string_id instead.

* Remove get_realm.

* (This commit) Rename get_realm_by_string_id to get_realm.

Part of a larger migration to remove the Realm.domain field entirely.
2017-01-04 17:12:23 -08:00
Rafid Aslam 50afe5a6c2 sync_api_key: Fix `ConfigParser` deprecation warnings.
Fix `ConfigParser` deprecation warnings in
`zilencer/management/commands/sync_api_key.py`.

Fixes #2727.
2016-12-30 11:52:22 -08:00
Rafid Aslam aece2c7c45 populate_db: Fix naive datetime warnings
Fix naive datetime warnings in
`zilencer/management/commands/populate_db.py`.

Fixes #2729.
2016-12-20 17:48:22 -08:00
Tim Abbott d3b1f6e273 populate_db: Add notifications stream for Zulip realm. 2016-12-15 22:43:08 -08:00
Rishi Gupta 6b39f3222c Remove Realm.domain from populate_db.py.
Fixes #2286.
2016-12-15 13:22:17 -08:00
Rishi Gupta fc188de8e0 bulk_create: Refactor bulk_create_users to take Realm instead of domains.
Previously bulk_create_users would figure out a user's realm from their
email domain. Now require that a realm be passed explicitly.
2016-12-15 13:22:17 -08:00
Rishi Gupta 56c8b58cbc populate_db: Remove create_streams.
Is extraneous indirection.
2016-12-15 13:22:17 -08:00
Rishi Gupta ee9b59321c populate_db: Refactor create_streams.
Remove unused field from stream_dict.
2016-12-15 13:22:17 -08:00
Rishi Gupta fbf48f7ec6 bulk_create: Refactor bulk_create_streams to not take domain dictionary.
First step in cleaning up populate_db.create_streams and
bulk_create.bulk_create_streams. Part of a series of commits to remove
Realm.domain from populate_db.
2016-12-15 13:22:17 -08:00
reyha 82e32ad255 Access realm by `string_id` in management commands.
`Realm.string_id` replaces 'Realm.domain'
in the management commands.

Fixes #2325.
2016-12-14 10:38:03 -08:00
Juan Verhook 9869fb8d87 Add UserPresence rows to test databases.
Fixes: #2426

mypy: Added annotations to populate_db.py
2016-12-09 15:41:21 -08:00
Joy Chen dd95a5e03f streams: Add default stream description population data. 2016-12-07 21:02:17 -08:00
anirudhjain75 beaa62cafa mypy: Convert several directories to use typing.Text.
Specifically, these directories are converted: [analytics/, scripts/,
tools/, zerver/management/, zilencer/, zproject/]
2016-12-07 20:51:05 -08:00
nikolay abc2ff4a06 pep8: Fix many rule E128 violations.
[Tweaked by tabbott to adjust some approaches used in wrapping]
2016-12-03 13:33:31 -08:00
bulat22101 a6f91064a2 pep8: Fix E129 violations 2016-12-03 10:56:36 -08:00
Alex Huang c8ddea16c3 pep8: Fix E122. 2016-12-01 23:16:35 -08:00
AZtheAsian 7e14fe65ab pep8: fix E202 violations 2016-12-01 23:06:02 -08:00
Rafid Aslam c5316b4002 lint: Fix E127 pep8 violations.
Fix pep8: E127 continuation line over-indented for visual indent
style issue.
2016-12-01 10:23:55 -08:00
Tommy Ip e4091c6413 pep8: Fix E222 violations. 2016-11-30 21:49:02 +00:00
Rishi Gupta 4b183cd526 domain migration: Remove several instances of get_realm.
Remove the easy to remove instances of get_realm.
2016-11-26 15:19:56 -08:00
Anders Kaseorg 207cf6302b Always start python via shebang lines.
This is preparation for supporting using Python 3 in production.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 14:46:37 -08:00
Steve Howell 78474c8bee Add test users with non-lowercase emails.
We are prone to case-sensitivity bugs, so I added AARON and ZOE.
Also, for good measure, I insert them in non-alphabetical order
to try to drive out bugs from non-consistent sorting of user ids.
2016-11-26 11:48:52 -08:00
Tim Abbott 61816f6151 populate_db: Remove replay_old_messages code.
Now that we have the data import/export tool, that's a better
mechanism for importing data than this old and likely buggy code was.
2016-11-15 22:34:44 -08:00
paxapy b90057d557 render_messages.py: render first version of edited messages.
This decreases the number of spurious differences in rendered content
significantly.
2016-11-15 13:39:11 -08:00
Rishi Gupta 5681ee90fa models.py: Rename split_email_to_domain to email_to_domain.
In order to better match models.email_to_username.
2016-11-11 15:26:51 -08:00
Rishi Gupta c1713c9659 Prevent code from using email domain to determine realm when subdomains.
Also removes the intermediate step of going through Realm.domain in the
non-subdomains case. Part of a larger project to remove Realm.domain
entirely.
2016-11-11 15:26:51 -08:00
Umair Khan e127325fce Django 1.10: Update zilencer urls. 2016-11-10 16:20:03 +05:00
paxapy 93965a8e89 Add tools for dumping and comparing markdown renderings.
This adds a couple new tools that can be used to determine whether a
particular change in Zulip's backend markdown processor would impact
the rendering of historical messages, without a human actually looking
at the message content.  This is a useful way to verify whether a
change to our markdown syntax is likely to create problems.

[commit message and code tweaked by tabbott]
2016-11-07 07:49:56 -08:00
Umair Khan 682aa1f298 Django 1.10: Use add_argument for options in BaseCommand. 2016-11-04 10:20:23 -07:00
Umair Khan b140236fcf Django 1.10: Do not use patterns function. 2016-11-04 10:06:00 -07:00
Rishi Gupta c0f1b5d9f6 Change Realm defaults to Community defaults.
Previously, we set restrict_to_domain and invite_required differently
depending on whether we were setting up a community or a corporate
realm. Setting restrict_to_domain requires validation on the domain of the
user's email, which is messy in the web realm creation flow, since we
validate the user's email before knowing whether the user intends to set up
a corporate or community realm. The simplest solution is to have the realm
creation flow impose as few restrictions as possible (community defaults),
and then worry about restrict_to_domain etc. after the user is already in.

We set the test suite to explictly use the old defaults, since several of
the tests depend on the old defaults.

This commit adds a database migration.
2016-11-03 13:53:01 -07:00
Steve Howell af6512e782 Add feedback bot to the test database. 2016-11-03 09:42:27 -07:00
Rishi Gupta db0e509422 do_create_realm: Replace domain argument with string_id.
Turns string_id into a required argument, and domain into an optional
argument.
2016-11-02 22:46:34 -07:00
Rishi Gupta 64bcd71d6e models.Realm: Rename subdomain to string_id.
Does a database migration to rename Realm.subdomain to
Realm.string_id, and makes Realm.subdomain a property.  Eventually,
Realm.string_id will replace Realm.domain as the handle by which we
retrieve Realm objects.
2016-11-02 22:46:34 -07:00
Tomasz Kolek 4790316b57 Add user setting option to always send push notifications.
Add option in user's settings for getting mobile push notifications
even if a Zulip browser is online.  Default is False.

Fixes: #1596.
2016-10-25 10:52:29 -07:00
Rishi Gupta 537ee75761 Create RealmAlias entries for Realm.domain.
This is a preliminary step towards eliminating the realm.domain field
in favor of realm.subdomain.  Includes a database migration to create
these for existing realms.
2016-10-25 10:06:39 -07:00