Commit Graph

185 Commits

Author SHA1 Message Date
reyha 82e32ad255 Access realm by `string_id` in management commands.
`Realm.string_id` replaces 'Realm.domain'
in the management commands.

Fixes #2325.
2016-12-14 10:38:03 -08:00
Juan Verhook 9869fb8d87 Add UserPresence rows to test databases.
Fixes: #2426

mypy: Added annotations to populate_db.py
2016-12-09 15:41:21 -08:00
Joy Chen dd95a5e03f streams: Add default stream description population data. 2016-12-07 21:02:17 -08:00
anirudhjain75 beaa62cafa mypy: Convert several directories to use typing.Text.
Specifically, these directories are converted: [analytics/, scripts/,
tools/, zerver/management/, zilencer/, zproject/]
2016-12-07 20:51:05 -08:00
nikolay abc2ff4a06 pep8: Fix many rule E128 violations.
[Tweaked by tabbott to adjust some approaches used in wrapping]
2016-12-03 13:33:31 -08:00
bulat22101 a6f91064a2 pep8: Fix E129 violations 2016-12-03 10:56:36 -08:00
Alex Huang c8ddea16c3 pep8: Fix E122. 2016-12-01 23:16:35 -08:00
AZtheAsian 7e14fe65ab pep8: fix E202 violations 2016-12-01 23:06:02 -08:00
Rafid Aslam c5316b4002 lint: Fix E127 pep8 violations.
Fix pep8: E127 continuation line over-indented for visual indent
style issue.
2016-12-01 10:23:55 -08:00
Tommy Ip e4091c6413 pep8: Fix E222 violations. 2016-11-30 21:49:02 +00:00
Rishi Gupta 4b183cd526 domain migration: Remove several instances of get_realm.
Remove the easy to remove instances of get_realm.
2016-11-26 15:19:56 -08:00
Anders Kaseorg 207cf6302b Always start python via shebang lines.
This is preparation for supporting using Python 3 in production.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2016-11-26 14:46:37 -08:00
Steve Howell 78474c8bee Add test users with non-lowercase emails.
We are prone to case-sensitivity bugs, so I added AARON and ZOE.
Also, for good measure, I insert them in non-alphabetical order
to try to drive out bugs from non-consistent sorting of user ids.
2016-11-26 11:48:52 -08:00
Tim Abbott 61816f6151 populate_db: Remove replay_old_messages code.
Now that we have the data import/export tool, that's a better
mechanism for importing data than this old and likely buggy code was.
2016-11-15 22:34:44 -08:00
paxapy b90057d557 render_messages.py: render first version of edited messages.
This decreases the number of spurious differences in rendered content
significantly.
2016-11-15 13:39:11 -08:00
Rishi Gupta 5681ee90fa models.py: Rename split_email_to_domain to email_to_domain.
In order to better match models.email_to_username.
2016-11-11 15:26:51 -08:00
Rishi Gupta c1713c9659 Prevent code from using email domain to determine realm when subdomains.
Also removes the intermediate step of going through Realm.domain in the
non-subdomains case. Part of a larger project to remove Realm.domain
entirely.
2016-11-11 15:26:51 -08:00
Umair Khan e127325fce Django 1.10: Update zilencer urls. 2016-11-10 16:20:03 +05:00
paxapy 93965a8e89 Add tools for dumping and comparing markdown renderings.
This adds a couple new tools that can be used to determine whether a
particular change in Zulip's backend markdown processor would impact
the rendering of historical messages, without a human actually looking
at the message content.  This is a useful way to verify whether a
change to our markdown syntax is likely to create problems.

[commit message and code tweaked by tabbott]
2016-11-07 07:49:56 -08:00
Umair Khan 682aa1f298 Django 1.10: Use add_argument for options in BaseCommand. 2016-11-04 10:20:23 -07:00
Umair Khan b140236fcf Django 1.10: Do not use patterns function. 2016-11-04 10:06:00 -07:00
Rishi Gupta c0f1b5d9f6 Change Realm defaults to Community defaults.
Previously, we set restrict_to_domain and invite_required differently
depending on whether we were setting up a community or a corporate
realm. Setting restrict_to_domain requires validation on the domain of the
user's email, which is messy in the web realm creation flow, since we
validate the user's email before knowing whether the user intends to set up
a corporate or community realm. The simplest solution is to have the realm
creation flow impose as few restrictions as possible (community defaults),
and then worry about restrict_to_domain etc. after the user is already in.

We set the test suite to explictly use the old defaults, since several of
the tests depend on the old defaults.

This commit adds a database migration.
2016-11-03 13:53:01 -07:00
Steve Howell af6512e782 Add feedback bot to the test database. 2016-11-03 09:42:27 -07:00
Rishi Gupta db0e509422 do_create_realm: Replace domain argument with string_id.
Turns string_id into a required argument, and domain into an optional
argument.
2016-11-02 22:46:34 -07:00
Rishi Gupta 64bcd71d6e models.Realm: Rename subdomain to string_id.
Does a database migration to rename Realm.subdomain to
Realm.string_id, and makes Realm.subdomain a property.  Eventually,
Realm.string_id will replace Realm.domain as the handle by which we
retrieve Realm objects.
2016-11-02 22:46:34 -07:00
Tomasz Kolek 4790316b57 Add user setting option to always send push notifications.
Add option in user's settings for getting mobile push notifications
even if a Zulip browser is online.  Default is False.

Fixes: #1596.
2016-10-25 10:52:29 -07:00
Rishi Gupta 537ee75761 Create RealmAlias entries for Realm.domain.
This is a preliminary step towards eliminating the realm.domain field
in favor of realm.subdomain.  Includes a database migration to create
these for existing realms.
2016-10-25 10:06:39 -07:00
Steve Howell d21d443d29 Deprecate render_old_messages command. 2016-10-04 11:31:21 -07:00
Steve Howell 7fb992dba3 Simplify and "fix" render_old_messages management command.
The command to render old messages now looks for all messages
not matching the bugdown version, and it no longer directly calls
into model code.  We should still be extremely cautious about
using this code.
2016-10-04 11:31:20 -07:00
hackerkid ea39fb2556 Add option for hosting each realm on its own subdomain.
This adds support for running a Zulip production server with each
realm on its own unique subdomain, e.g. https://realm_name.example.com.

This patch includes a ton of important features:
* Configuring the Zulip sesion middleware to issue cookier correctly
  for the subdomains case.
* Throwing an error if the user tries to visit an invalid subdomain.
* Runs a portion of the Casper tests with REALMS_HAVE_SUBDOMAINS
  enabled to test the subdomain signup process.
* Updating our integrations documentation to refer to the current subdomain.
* Enforces that users can only login to the subdomain of their realm
  (but does not restrict the API; that will be tightened in a future commit).

Note that toggling settings.REALMS_HAVE_SUBDOMAINS on a live server is
not supported without manual intervention (the main problem will be
adding "subdomain" values for all the existing realms).

[substantially modified by tabbott as part of merging]
2016-09-27 23:24:14 -07:00
Sahil Dua 058587da77 Remove extra new lines at the ends of Zulip authoried files.
Fixes #1627.

[tweaked by tabbott to avoid patching third-party modules, for now]
2016-09-26 21:05:24 -07:00
Tomasz Kolek 0e192d6aff Add colors to streams in development server fixtures.
Fixes: #1454.
2016-08-26 22:38:14 -07:00
Taranjeet Singh 230cc228c2 profile_request.py: Fix line with length greater than 120.
This updates file zilencer/management/commands/profile_request.py.
2016-08-19 11:56:43 -07:00
Rishi Gupta 708b416ca1 Terms of Service: Fix corner cases around new users being created.
This fixes a few bugs in 7910a6e134,
related to automatically created user accounts.
2016-08-13 00:19:54 -07:00
Tim Abbott 95aee366c8 Enable local browser error reporting if ZILENCER_ENABLED. 2016-08-11 13:49:18 -07:00
Tim Abbott ba7ef1c82d zilencer: Fix regression in URLs.
When we refactored zilencer to use a single urls.py file in
bf50dd7771, we accidentally lost the
prefix on the API urls.

This broke sending error report emails if zilencer was enabled.
2016-08-11 12:38:42 -07:00
Tim Abbott 0ca7af6137 Annotate zilencer tests. 2016-08-04 15:53:23 -07:00
Tim Abbott 1f2b3588d2 Annotate zilencer/models.py and fix str types in zilencer views. 2016-08-04 15:53:23 -07:00
Taranjeet Singh 0cafd9268d zilencer/management/commands/populate_db.py: Fix line with len > 120. 2016-07-18 14:42:39 -07:00
Eklavya Sharma 90f306421f Replace bytes by strings in zilencer/migrations. 2016-07-04 12:08:37 -07:00
Eklavya Sharma 88354ad1fc zilencer's populate_db.py: Remove unnecessary cast to text_type. 2016-06-29 10:36:50 -07:00
Eklavya Sharma e155573a1c zilencer's populate_db.py: Type annotate all variables. 2016-06-27 10:22:49 -07:00
Eklavya Sharma 8cacd5fcf8 zilencer's populate_db.py: Type annotate some variables. 2016-06-27 21:00:10 +05:30
Eklavya Sharma ea3c671494 zilencer's populate_db.py: Improve an annotation. 2016-06-27 21:00:10 +05:30
Eklavya Sharma 66d1867869 zilencer's populate_db.py: Fix string types. 2016-06-27 21:00:10 +05:30
acrefoot e4ed9195dc Remove rest_dispatch hack and optimize imports.
For a long time, rest_dispatch has had this hack where we have to
create a copy of it in each views file using it, in order to directly
access the globals list in that file.  This removes that hack, instead
making rest_dispatch just use Django's import_string to access the
target method to use.

[tweaked and reorganized from acrefoot's original branch in various
ways by tabbott]
2016-06-24 16:11:03 -07:00
Tomasz Kolek f8c368c07f Add default-bot during populate_db. 2016-06-21 11:47:38 -07:00
Eklavya Sharma d169cc5376 zerver/lib/bulk_create.py: Fix string annotations.
Change string type from `str` to `text_type` where required.
Also fix clashing annotations.
2016-06-13 10:08:14 +05:30
Tomasz Kolek 2ac9c792f3 test_hooks: Use an incoming webhook bot for webhook tests. 2016-06-07 21:27:04 -07:00
Evan Palmer bd0fd61821 Annotate zilencer.error_notify.
Also fix the annotation of zilencer.views.report_error.

The `report` arguments are a Dict containing both strings and the
`more_info` sub-dictionary, so we type them as Dict[str, Any].

[tweaked by tabbott]
2016-06-07 13:44:56 -07:00
pxhanus e247d9783f Annotate zilencer/views.py. 2016-06-04 22:07:01 -07:00
Tim Abbott f2320bf27f Fix populate_db send_messages annotation. 2016-06-04 16:11:14 -07:00
medullaskyline 7c2c7fb31c Annotate zerver/lib/bulk_create.py. 2016-06-04 15:51:05 -07:00
Tim Abbott a1a27b1789 Annotate most Zulip management commands. 2016-06-04 10:12:06 -07:00
Ashish Kumar 31bf6b8259 Type annotation of zerver/models.py
[Substantially revised by tabbott]

This probably still has some bugs in it, but having mostly complete
annotations for models.py will help a lot for the annotations folks
are adding to other files.
2016-06-02 23:28:34 -07:00
Umair Khan 08fbd57245 [i18n] Make error messages translatable.
Make all strings passing through `json_error` and `JsonableError`
translatable.

Fixes #727
2016-05-31 07:40:42 -07:00
Vishnu Ks 100d885f23 Change default announcement stream to announce.
Fixes #788.
2016-05-31 07:38:07 -07:00
Tim Abbott 960144a49e Desupport using uninstantiated REQ with has_request_variables.
This makes life difficult for doing static type annotations, and
didn't make the code look that much better anyway.
2016-05-31 07:31:15 -07:00
Eklavya Sharma 94e4b39112 Replace python2.7 by python everywhere. 2016-05-29 05:03:08 -07:00
Tomasz Kolek 8c18b8947f Add bot_type field to UserProfile.
This is intended to support creating different types of bots with
potentially limited permissions.
2016-05-19 22:37:37 -07:00
Umair Khan f9bbc5d6ff Enable i18n support in URL configuration.
This supports i18n using all of the following:
- I18N urls
- Session
- Cookie
- HTTP header
2016-05-19 08:33:30 -07:00
Tim Abbott bf50dd7771 Simplify zilencer urls.py configuration to a single file.
The previous separated-out configuration wasn't helping us, and this
makes it easier to make the extra installed applications pluggable in
the following commits.
2016-04-26 21:35:07 -07:00
Tim Abbott 49799440a4 Replace use of django-guardian with fields on UserProfile.
As documented in https://github.com/zulip/zulip/issues/441, Guardian
has quite poor performance, and in fact almost 50% of the time spent
running the Zulip backend test suite on my laptop was inside Guardian.

As part of this migration, we also clean up the old API_SUPER_USERS
variable used to mark EMAIL_GATEWAY_BOT as an API super user; now that
permission is managed entirely via the database.

When rebasing past this commit, developers will need to do a
`manage.py migrate` in order to apply the migration changes before the
server will run again.

We can't yet remove Guardian from INSTALLED_APPS, requirements.txt,
etc. in this release, because otherwise the reverse migration won't
work.

Fixes #441.
2016-04-20 21:51:52 -07:00
Tim Abbott 26463bb34d Fix nondeterministic subscriptions for default test users.
Previously, the UserProfile objects were created in the order
generated by a Set, which meant tests would randomly start failing if
the code that runs before this part of populate_db changed (and thus
caused the Set object used to pass users into bulk_create_users to
have a different order when enumerated).

This fixes the issue in two ways -- one by sorting the users inside
bulk_create_users, and second by attaching subscriptions to users
based on a deterministic ordering.
2016-04-13 13:19:02 -07:00
Tim Abbott a1b306f9ce Finish purging 'fromt typing import *' from Zulip codebase. 2016-04-07 14:11:21 -07:00
Tim Abbott b7dcf2181f Add PEP-484 type annotations to management commands. 2016-04-03 15:40:23 -07:00
Tim Abbott b99313545e Add PEP-484 type annotations to global dictionaties. 2016-04-03 15:40:06 -07:00
Eklavya Sharma 9e3c3e14f5 Partially apply Python 3 libmodernize.fixes.fix_dict_six.
Refer to #256
2016-03-19 15:52:58 -07:00
Varshit 72033069ed Extend lint-all to check for newlines at the end of files. 2016-03-17 23:03:56 -07:00
Tim Abbott 5f9cd4d7c8 populate_db: Refactor saved_data to use a consistent type. 2016-02-03 19:29:45 -08:00
Tim Abbott c55ac01ae6 populate_db: Rename confusing sub local variable.
While the code was technically correct, elsewhere in that function sub
a Subscription object, not a tuple of ints.
2016-02-03 19:29:45 -08:00
Tim Abbott a2ef1642d1 Fix buggy report_error return value. 2016-02-03 19:29:44 -08:00
Tim Abbott 5bacda3662 python3: Fix usage of .keys()/.values() to handle iterators.
This fixes the places where we use the result of .keys(), .items(),
and .values() that wouldn't work with an iterator to wrap them with
list().
2016-01-26 21:11:25 -08:00
Tim Abbott 1f44417fc1 Switch to using Python 3 style division everywhere.
Also add testing for this to our Python 3 compatibility test suite.
2016-01-26 21:09:43 -08:00
Tim Abbott 700055c194 Apply modernize transform libmodernize.fixes.fix_file.
This replaces use of file() with open() which is python 3 compatible,
and also adds it to our python 3 support test suite.
2016-01-26 21:09:42 -08:00
Tim Abbott a79e89b28f Cleanup remaining usage of % comprehensions without explicit tuples. 2015-12-05 15:29:42 -08:00
Tim Abbott 16ae985807 profile_request: Fix get_old_messages_backend import. 2015-11-11 21:34:39 -08:00
Tim Abbott f7878a61e1 Apply Python 3 futurize transform libmodernize.fixes.fix_xrange_six. 2015-11-01 09:35:06 -08:00
Tim Abbott 2ea0663a4a Apply Python 3 futurize transform libmodernize.fixes.fix_imports_six. 2015-11-01 09:26:16 -08:00
Tim Abbott f3783fb4a1 Apply Python 3 futurize transform libfuturize.fixes.fix_print_with_import. 2015-11-01 09:26:16 -08:00
Tim Abbott 9c66229456 Apply Python 3 futurize transform libfuturize.fixes.fix_absolute_import. 2015-11-01 09:26:16 -08:00
Steven Oud d5435fad1d Consistently use /usr/bin/env python2.7 in shebangs and commands. 2015-10-21 22:58:21 +00:00
Tim Abbott 71a06d58de Convert uses of Realm.objects.get() to get_realm().
get_realm is better in two key ways:
* It uses memcached to fetch the data from the cache and thus is faster.
* It does a case-insensitive query and thus is more safe.
2015-10-15 09:16:58 -04:00
Tim Abbott 792075ddeb sync_api_key: Don't throw a messy exception when user doesn't exist. 2015-10-15 09:16:58 -04:00
Tim Abbott 3b84a9a010 Modernize management command for deactivating a realm.
(imported from commit 9ef681193ff04e447c0afbba66187357624d98f3)
2015-09-19 19:10:27 -07:00
Reid Barton 9eee250720 Give the realm created by populate_db a name
I got tired of seeing the tab title "null - Dropbox Chat".

(imported from commit fb88bf1e01b0d0ecbfe2320abb02a69afbb2ac38)
2015-08-21 17:02:47 -07:00
Tim Abbott a8261dd6ac Remove Deployment initialization from Development.
(imported from commit 4596fcf480c10c88496cd6506edb0a6e702d823c)
2015-08-21 10:40:57 -07:00
David Roe edf7e732a2 ENTERPRISE => VOYAGER.
(imported from commit 4f8080b9f506a87ca40bef32e39de5218cba916a)
2015-08-21 10:33:35 -07:00
Reid Barton ae0ae3dde8 Django 1.8: declare positional arguments in management commands
(imported from commit d9efca1376de92c8187d25f546c79fece8d2d8c6)
2015-08-20 23:35:40 -07:00
Reid Barton 599742536b Switch from South to native Django migrations
This commit loses some indexes, unique constraints etc. that were
manually added by the old migrations. I plan to add them to a new
migration in a subsequent commit.

(imported from commit 4bcbf06080a7ad94788ac368385eac34b54623ce)
2015-08-20 23:01:26 -07:00
Anders Kaseorg 95afea9006 Remove enterprise registration form
(imported from commit 58914685752bff3c9d0e56ffe21bd3cd367e70e7)
2015-08-20 18:40:34 -07:00
David Roe 4bd93325de Don't create the MIT realm in the Dev VM.
(imported from commit 2e8122cfca87ace24f5048a6ecc57c8c5b3d101c)
2015-08-19 22:24:53 -07:00
David Roe a9023935e4 Make the default Dev VM user Iago a Zulip admin.
(imported from commit ebec7b44c1d7fdbd270ee534f2c5453a21ae3884)
2015-08-19 22:24:53 -07:00
Tim Abbott dcc3b3fe37 Don't create Zulip employee accounts automatically in Dev VM.
(imported from commit 378f12d3f2ac4c131919c0efbd049b9eaa961694)
2015-08-18 20:46:35 -07:00
Zev Benjamin 517703ab47 Update default api/site URLs to prod
(imported from commit bcd32a0e8f197abd728616d1b0ae1c78a6639e68)
2015-02-05 22:46:31 -08:00
Jessica McKellar 4e24b432c4 [manual] Add script to migrate subs to new stream notification settings.
Previously, streams used `notifications`. Now they have separate
`desktop_notifications` and `audible_notifications`.

On staging, this should be run after the schema migration has been
applied (and technically before the code that uses the new fields is
deployed, but for staging, processing our notification settings a few
minutes late won't hurt anyone).

On prod, the script should be re-run just before the code using the
new notification settings is deployed to process any customer
notification settings changed since the staging deploy.

(imported from commit d99d238cd1b317c5180d7f940d70a7e2f8f9c712)
2014-03-03 16:08:33 -05:00
Steve Howell ebce82b136 Eliminate json_to_dict and use check_dict instead.
All usages of json_to_dict were replaced with the check_dict
validator.  The check_dict validations can eventually be
extended to validate the keys and values of incoming data,
but now we just use check_dict([]) in all the places where
we had json_to_dict, which means we aren't checking for any
specific keys; we are just making sure it's a dictionary.

(imported from commit fc5add9a7ef149dfac2a9a6d9a153799c4c0c24d)
2014-02-18 13:02:08 -05:00
Zev Benjamin 6691789cce Add helper function for returning a Redis client
(imported from commit 47f87d388d24343ac6b631181a55287eb8cd4a6d)
2014-02-10 13:23:28 -05:00
Steve Howell 8b09d4da6c Track feedback sender times in redis.
An earlier commit today made it so that we don't create tickets
for feedback if the Python process had seen a message from that
user in the last few minutes.  This doesn't help much when you
have multiple processes, so now we track the times in redis.

(imported from commit 62ff8ceb55d815d03566f407c1c85037334e4d6d)
2014-01-17 13:39:31 -05:00
Steve Howell 4477ee4bba Avoid excessive feedback ticket headers.
(imported from commit 84eb0945e23f670f4d343672dc5589a1867926d6)
2014-01-17 09:59:45 -05:00