Commit Graph

1431 Commits

Author SHA1 Message Date
Vishnu Ks 8a46ddf9d7 dev_settings: Add localhost:9991 as realm host for zulip realm.
[greg: Tweaked explanatory comment.]
2017-11-14 18:34:17 -08:00
Steve Howell ae5ba7f4fd Add a couple cross-realm bots.
These are new:

    new-user-bot
    emailgateway

Our cross-realm bots are hard coded to have email addresses
in the `zulip.com` domain, and they're not part of ordinary
realms.

These have always been cross-realm, but new enforcement in the
frontend code of all messages having been sent by a known user means
that it's important to add these properly.
2017-11-14 16:02:19 -08:00
Greg Price 415da352df prod install: Cut premature suggestion to configure auth backends.
The warning here means that the admin can't really act on this yet if
they want to disable email auth, which is likely among admins that
want to make any changes here.  And for admins who don't, this is an
extra thing to read and make a decision about before they can get a
server running.  See #6985.

Conversely, we already discuss auth backends right at the top of the
`prod-customize` doc, which is linked under "Next steps" at the end of
these instructions.

The warning about EmailAuthBackend is important; but we can move it to
the config file right next to the setting, and then it's available
right when it's actionable, which is if the admin is actually thinking
about changing the setting.
2017-11-14 15:57:39 -08:00
Vishnu Ks 36f29764cb emails: Add option to forward mails send in dev env to external email.
Fixes #7085.
2017-11-14 15:27:29 -08:00
Vishnu Ks 2af249dd5d api: Add description attribute to default stream group. 2017-11-14 14:41:42 -08:00
Vishnu Ks f45ba7de93 api: Pass group id instead of name to default stream group api. 2017-11-14 14:41:42 -08:00
Umair Khan 0e21cbc5d9 user-groups: Create static/js/user_groups.js. 2017-11-13 13:08:15 +05:00
derAnfaenger d707e3bc1d bots: Add USER_STATE_SIZE_LIMIT setting. 2017-11-10 18:56:26 -08:00
Tim Abbott 54aa87fba3 api: Migrate to using the new version of the API site. 2017-11-10 17:56:22 -08:00
Vishnu Ks 4a102b248a droplets: Set EXTERNAL_HOST to username.zulipdev.org:9991. 2017-11-10 16:13:53 -08:00
Vishnu Ks f44b60a150 Implement API for default stream groups. 2017-11-10 16:05:36 -08:00
Brock Whittaker 00d11fb0af input-pill: Add styling.
This adds some styling for the default case of the input pills and some
custom styling.
2017-11-10 14:14:03 -08:00
Brock Whittaker 7b00736fa2 input-pill: Add "input_pill" class and documentation. 2017-11-10 14:14:03 -08:00
Cynthia Lin 0b800b0a7d icons: Create framework for custom icons and add new bot icon. 2017-11-10 11:18:42 -08:00
Umair Khan b1603d289c user-groups: Add update members API endpoint.
Significantly modified by tabbott to fix some bugs.
2017-11-09 17:35:37 -08:00
Umair Khan a1b894a786 user-groups: Add delete API endpoint.
Significantly modified by tabbott to fix some bugs.
2017-11-09 17:35:30 -08:00
Umair Khan eeaf0f1742 user-groups: Add edit API endpoint.
Significantly modified by tabbott to fix some bugs.
2017-11-09 17:35:22 -08:00
Umair Khan 1bbe84af49 user-groups: Add create API endpoint.
Significantly modified by tabbott for better security structure.
2017-11-09 17:26:14 -08:00
Tim Abbott 900cd13865 docs: Link to postgres documentation for SSL modes.
For some historical reason we'd had the Postgres documentation on
valid SSL modes copied into the Zulip settings.py template file.  This
fixes that historical artifact.
2017-11-07 16:55:32 -08:00
rht 047eca1629 zproject: Remove unused imports (F401). 2017-11-07 16:37:03 -08:00
rht ccf2792c1c refactor: Remove six.moves.configparser import. 2017-11-07 10:51:44 -08:00
Steve Howell f105c480b3 Calculate gravatar hashes on the webapp client.
This change prepares us to have the server send avatar_url
of None when somebody wants a gravatar avatar (as opposed
to a user-uploaded one).

Subsequent commits will change behavior on both the server
and client to have this happen.  So this commit has no-op
code for now, but it will soon use the fallback-to-gravatar
logic.
2017-11-07 10:36:02 -08:00
rht e33c8d91e0 zproject: Remove inheritance from object. 2017-11-06 08:53:48 -08:00
rht 26b6b893e6 zproject: Use python 3 syntax for typing. 2017-11-04 19:58:03 -07:00
neiljp (Neil Pilgrim) 1197ff9655 mypy: Explicitly return Set[Any] for empty set in backends.py. 2017-11-04 19:47:45 -07:00
Greg Price 909631b5a9 docs/oauth: Update for Google UI changes, and for zulipdev.com.
The control panel on the Google side doesn't seem to match the
instructions we have; it looks pretty 2017 to me, so I imagine
it's had a redesign since the instructions were written.

Also, in dev, EXTERNAL_HOST is now a port on zulipdev.com, not on
localhost.

Update these instructions for those developments, and edit lightly.
In dev, recommend setting in `dev_settings` instead of in
`prod_settings_template`; that feels to me a little more reflective of
the actual intent, and the effect should be equivalent.
2017-11-03 13:38:59 -07:00
Greg Price 119bc14182 alias domains: Add a v1 of this feature.
The main limitation of this version is that it's controlled entirely
from settings, with nothing in the database and no web UI or even
management command to control it.  That makes it a bit more of a
burden for the server admins than it'd ideally be, but that's fine
for now.

Relatedly, the web flow for realm creation still requires choosing a
subdomain even if the realm is destined to live at an alias domain.

Specific to the dev environment, there is an annoying quirk: the
special dev login flow doesn't work on a REALM_HOSTS realm.  Also,
in this version the `add_new_realm` and `add_new_user` management
commands, which are intended for use in development environments only,
don't support this feature.

In manual testing, I've confirmed that a REALM_HOSTS realm works for
signup and login, with email/password, Google SSO, or GitHub SSO.
Most of that was in dev; I used zulipstaging.com to also test
 * logging in with email and password;
 * logging in with Google SSO... far enough to correctly determine
   that my email address is associated with some other realm.
2017-11-03 13:38:59 -07:00
Greg Price b33d981d7b settings: Simplify setting ALLOWED_HOSTS, by handling EXTERNAL_HOST.
This means one fewer thing the admin typically needs to read, absorb,
and make a decision about at install time.

The one way this change could hypothetically cause trouble is if the
admin wants to keep subdomains of EXTERNAL_HOST out of ALLOWED_HOSTS.
But while the subdomains often won't exist as domain names, it's hard
to imagine the situation in which they would exist but be under
someone else's control, or be doing something other than serving
Zulip realms.
2017-11-03 13:38:59 -07:00
rht 38acddee99 zproject: Remove u prefix from strings. 2017-11-02 11:01:47 -07:00
Steve Howell 85f605a978 Fix bug with EmailLogBackend.send_messages.
The `logging.info` call was not properly using string interpolation.
2017-11-01 10:05:13 -07:00
Tim Abbott d27002ba21 landing-page: Replace /about with /team and /history in links. 2017-10-31 12:44:05 -07:00
Brock Whittaker 425738fa24 landing-page: Extract /team/ from /about/.
This extracts and creates a /team/ page using the existing
'team.html' template and applying the why-page styling to it.
2017-10-31 11:37:42 -07:00
Brock Whittaker e86e259cb1 /api/: Add Markdown-based /api-new documentation.
This commit allows for the /api-new/ page to rendered similarly to our
/help pages.  It's based on the old content for /api, but we're not
replacing the old content yet, to give a bit of time to restructure
things reasonably.

Tweaked by eeshangarg and tabbott.
2017-10-30 22:32:43 -07:00
Greg Price 271a9f0da7 settings: Eliminate disused variable EXTERNAL_API_URI.
This was part of the logic to handle EXTERNAL_API_PATH varying.
But also it was already no longer used -- it was only ever passed
into template contexts, as `external_api_uri`, and it'd been
overtaken there by `external_api_uri_subdomain`.

So, update our dev docs to reflect that, and eliminate the variable.
2017-10-30 18:29:29 -07:00
Greg Price 7366a4ca7a settings: Eliminate the disused EXTERNAL_API_PATH.
This setting isn't documented at all, and I believe nobody has used it
since the end of api.zulip.com in 2016.  So we get to complete the
cleanup of this logic.
2017-10-30 18:29:29 -07:00
rht c4fcff7178 refactor: Replace super(.*self) with Python 3-specific super().
We change all the instances except for the `test_helpers.py`
TimeTrackingCursor monkey-patching, which actually needs to specify
the base class.
2017-10-30 14:30:25 -07:00
Brock Whittaker 6933d51c0f views/integrations: Change non-generic HelpView to MarkdownDirectoryView.
The HelpView class will render a directory as markdown with an index HTML
page. This however can also be used for other generics and applied to
the API pages as well, so change the class to a generic class and
specify the path templates and names.

Tweaked by tabbott and Eeshan Garg.
2017-10-29 18:15:36 -07:00
Harshit Bansal 56f26c2d6f zproject/settings.py: Correct emoji sprite CSS files. 2017-10-28 10:38:33 -07:00
Tim Abbott f790e667b3 test_tornado: Fix repeated autoreload configuration in tests.
Apparently, our previous efforts to disable AUTORELOAD in the tests
were incorrect.
2017-10-27 16:19:03 -07:00
Tim Abbott e5df05fd35 tests: Suppress logging spam in email mirror tests. 2017-10-27 16:06:03 -07:00
Tim Abbott ef9679667f test_docs: Fix unnecessary info-level logging in tests. 2017-10-27 15:44:45 -07:00
Tim Abbott e19312b394 Fix zerver.lib.digest log being printed in tests. 2017-10-27 15:44:15 -07:00
Tim Abbott e8ab7cd1a9 logging: Fix zulip.send_email log being printed in tests.
Our set_loglevel tool didn't set propagate to False, so just using it
directly wouldn't work unless the logger is explcitly declared in
zproject/settings.py, which this one isn't.
2017-10-27 15:34:36 -07:00
Greg Price 318682fd52 auth: Use URL rather than cookie to pass signed data cross-domain.
The cookie mechanism only works when passing the login token to a
subdomain.  URLs work across domains, which is why they're the
standard transport for SSO on the web.  Switch to URLs.

Tweaked by tabbott to add a test for an expired token.
2017-10-27 14:42:04 -07:00
Greg Price fad3d56810 views: Move some login code from `registration` to `auth`.
Most of these have more to do with authentication in general than with
registering a new account.  `create_preregistration_user` could go
either way; we move it to `auth` so we can make the imports go only in
one direction.
2017-10-27 14:28:38 -07:00
Henrik Pettersson 09cd47c6ad Add UI for viewing and cancelling open Zulip invitations.
Lets administrators view a list of open(unconfirmed) invitations and
resend or revoke a chosen invitation.

There are a few changes that we can expect for the future:

  * It is currently possible to invite an email that you have already
    invited, it might make sense to change this behavior.

  * Resend currently sends an invite reminder instead of resending the
    original invite, this is because 'custom_body' was not stored when
    the first invite was sent.

Tweaked in various minor ways, primarily in the backend, by tabbott,
mostly for style consistency with the rest of the codebase.

Fixes: #1180.
2017-10-27 13:07:43 -07:00
Tim Abbott 8e2cdedf9a lint: Fix lines in Python codebase longer than 120 characters. 2017-10-26 17:47:30 -07:00
Tim Abbott b936e8c24b lint: Fix lines in Python codebase longer than 125 characters. 2017-10-26 17:36:54 -07:00
Tim Abbott be619fe881 lint: Wrap many very long lines in the Python codebase.
This decreases the maximum line length in our Python codebase to 130.
2017-10-26 17:31:58 -07:00
Greg Price 30cc2994de social auth: Replace a bit of explicit model-querying with get_realm. 2017-10-26 10:29:17 -07:00
Greg Price 27adbe8d79 subdomains: Clean up a use of various falsy values for the root domain.
This isn't a complete cleanup of the logic at this spot, but
at least the messy part that remains is now explicit.
2017-10-26 10:29:17 -07:00
Greg Price c9457d4af0 subdomains: Refactor check_subdomain to a clearer interface.
Now that every call site of check_subdomain produces its second
argument in exactly the same way, push that shared bit of logic
into a new wrapper for check_subdomain.

Also give that new function a name that says more specifically what
it's checking -- which I think is easier to articulate for this
interface than for that of check_subdomain.
2017-10-26 10:29:17 -07:00
Abhijeet Kaur a88178afaf Embedded bots: Add support for creating embedded bots via the API.
Adds support to add "Embedded bot" Service objects. This service
handles every embedded bot.

Extracted from "Embedded bots: Add support to add embedded bots from
UI" by Robert Honig.

Tweaked by tabbott to be disabled by default.
2017-10-25 16:13:29 -07:00
Vishnu Ks 9314a7ac8b backends: Move EmailLogBackEnd to email_backends. 2017-10-25 14:35:12 -07:00
Vishnu Ks 1d94119d31 actions: Call send_initial_pms from process_new_human_user. 2017-10-25 14:14:59 -07:00
Tim Abbott b590cd6c8f password-reset: Remove unnecessary template arguments.
We set these directly in the `send_email` function anyway.
2017-10-24 12:07:43 -07:00
Tim Abbott d91e49b681 settings: Fix whitespace in new LDAP settings code. 2017-10-24 11:47:01 -07:00
Tim Abbott 8a588ba23d ldap: Default REGISTER_LINK_DISABLED=False when using LDAP backend.
This should help prevent confusion where new users find themselves on
the LDAP login form and click "register" because they know they don't
have an account.  Whereas in fact, their account will be auto-created
if they just login, so there's no need for them to access it.
2017-10-24 11:21:49 -07:00
Tim Abbott 70d509196d backends: Call send_initial_pms on other user creation paths.
This fixes a problem we've seen where LDAP users were not getting this
part of the onboarding process, and a similar problem for human users
created via the API.

Ideally, we would have put these fixes in process_new_human_user, but
that would cause import loop problems.
2017-10-24 09:03:53 -07:00
Tim Abbott 716c525389 backends: Sort imports. 2017-10-24 08:59:39 -07:00
Tim Abbott ad165a6f8f settings: Remove remaining DEPLOYMENT_ROLE_* code remnants.
These should have been removed when we removed Zilencer.
2017-10-23 21:15:03 -07:00
darshanime 6078f648cb settings: use None for memcached cache timeout.
This is a cleaner approach than the old "really big number" strategy.
2017-10-23 16:34:43 -07:00
Maarten Rijke 841f02934d push_notifications: Add setting to redact content.
This commit adds a setting that toggles redacting content of push
notifications and replacing it with "***REDACTED***".
2017-10-19 18:16:06 -07:00
Rishi Gupta a08543f1ce server settings: Add setting for MAILING_ADDRESS.
The rules here are fuzzy, and it's quite possible none of Zulip's emails
need an address at all. Every country has its own rules though, which makes
it hard to tell. In general, transactional emails do not need an address,
and marketing emails do.
2017-10-19 00:04:07 -07:00
Tim Abbott 1ab2ca5986 subdomains: Extract zerver.lib.subdomains library.
These never really belonged with the rest of zerver.lib.utils.py, and
having a separate library makes it easier to enforce full test
coverage.
2017-10-18 22:27:48 -07:00
Aastha Gupta 0fae83b301 notifications: Prompt user to enable desktop notifications.
This is a two-step notifications process that will ask a user
to enable notifications and if they click exit give them three
options:

1. Enable notifications.
2. Ask later.
3. Never ask on this computer again.

The first two are self-explanatory (ask later = next session it
asks again). The third is captured and stored in localStorage and
a check is done on page load to see whether or not notifications
should be displayed.

Commit modified heavily by Brock Whittaker <brock@zulipchat.com>.

Fixes #1189.
2017-10-18 21:55:43 -07:00
Tim Abbott 1d503ad60c urls: Move the API URLs above the random other endpoints.
This should make it a lot easier for folks to find the Zulip API when
reading our urls.py file.
2017-10-17 22:09:22 -07:00
Alena Volkova 5515a075ec urls: Move the report endpoints to be API-style routes. 2017-10-17 22:05:56 -07:00
Greg Price 37f01add7e errors: Point to upstream 400-handling bug in our workaround. 2017-10-16 11:17:10 -07:00
Umair Khan a48a86237d ldap: Change logging level to warning.
Fixes #6960.
2017-10-13 17:13:18 -07:00
Umair Khan 490515aea6 cleanup: Fix comment in SocialAuthMixin.auth_complete. 2017-10-13 17:13:18 -07:00
Greg Price b095463f9b logging: Clear out the Django default config on logger `django`.
By default, Django sets up two handlers on this logger, one of them
its AdminEmailHandler.  We have our own handler for sending email on
error, and we want to stick to that -- we like the format somewhat
better, and crucially we've given it some rate-limiting through
ZulipLimiter.

Since we cleaned out our logging config in e0a5e6fad, though, we've
been sending error emails through both paths.  The config we'd had
before that for `django` was redundant with the config on the root --
but having *a* config there was essential for causing
`logging.config.dictConfig`, when Django passes it our LOGGING dict,
to clear out that logger's previous config.  So, give it an empty
config.

Django by default configures two loggers: `django` and
`django.server`.  We have our own settings for `django.server`
anyway, so this is the only one we need to add.

The stdlib `logging` and `logging.config` docs aren't 100% clear, and
while the source of `logging` is admirably straightforward the source
of `logging.config` is a little twisty, so it's not easy to become
totally confident that this has the right effect just by reading.
Fortunately we can put some of that source-diving to work in writing
a test for it.
2017-10-12 22:45:14 -07:00
Greg Price 55426894cd errors: Force a super-simpler handler for 400 errors.
This works around a bug in Django in handling the error case of a
client sending an inappropriate HTTP `Host:` header.  Various
internal Django machinery expects to be able to casually call
`request.get_host()`, which will attempt to parse that header, so an
exception will be raised.  The exception-handling machinery attempts
to catch that exception and just turn it into a 400 response... but
in a certain case, that machinery itself ends up trying to call
`request.get_host()`, and we end up with an uncaught exception that
causes a 500 response, a chain of tracebacks in the logs, and an email
to the server admins.  See example below.

That `request.get_host` call comes in the midst of some CSRF-related
middleware, which doesn't even serve any function unless you have a
form in your 400 response page that you want CSRF protection for.
We use the default 400 response page, which is a 26-byte static
HTML error message.  So, just send that with no further ado.

Example exception from server logs (lightly edited):

  2017-10-08 09:51:50.835 ERR  [django.security.DisallowedHost] Invalid HTTP_HOST header: 'example.com'. You may need to add 'example.com' to ALLOWED_HOSTS.
  2017-10-08 09:51:50.835 ERR  [django.request] Internal Server Error: /loginWithSetCookie
  Traceback (most recent call last):
    File ".../django/core/handlers/exception.py", line 41, in inner
      response = get_response(request)
    File ".../django/utils/deprecation.py", line 138, in __call__
      response = self.process_request(request)
    File ".../django/middleware/common.py", line 57, in process_request
      host = request.get_host()
    File ".../django/http/request.py", line 113, in get_host
      raise DisallowedHost(msg)
  django.core.exceptions.DisallowedHost: Invalid HTTP_HOST header: 'example.com'. You may need to add 'example.com' to ALLOWED_HOSTS.

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File ".../django/core/handlers/exception.py", line 109, in get_exception_response
      response = callback(request, **dict(param_dict, exception=exception))
    File ".../django/utils/decorators.py", line 145, in _wrapped_view
      result = middleware.process_view(request, view_func, args, kwargs)
    File ".../django/middleware/csrf.py", line 276, in process_view
      good_referer = request.get_host()
    File ".../django/http/request.py", line 113, in get_host
      raise DisallowedHost(msg)
  django.core.exceptions.DisallowedHost: Invalid HTTP_HOST header: 'example.com'. You may need to add 'example.com' to ALLOWED_HOSTS.
2017-10-11 20:43:36 -07:00
Tim Abbott 9426ee024b logging: Filter tornado.general logging for being spammy.
Apparently, this sockjs.tornado logging code resulted in a lot of
buggy error emails whenever a Zulip browser tried to reconnect on a
new IP.  I don't see an obvious way to suppress them from within
sockjs, but that might be a good follow-up issue.

Fixes #6959.
2017-10-11 16:53:52 -07:00
Brock Whittaker df9092cf42 components: Split out "app_components.css" for shared components.
This new “app_components.css” is for shared components between the
internal app and the portico pages.
2017-10-11 12:24:44 -07:00
Alena Volkova 79560e21bf urls: Move the json/tutorial_status endpoint to be an API-style route. 2017-10-09 15:13:33 -07:00
Tim Abbott 88bb6c6cad uploads: Set a BOTO_CONFIG path to override broken GCE configuration.
The comment is pretty self-explanatory.  The fact that Google Compute
Engine has this problem does not impress confidence about their
product, but hopefully this is the only really dumb thing they do.

Fixes #4839.
2017-10-08 19:46:57 -07:00
Greg Price 0691724836 passwords: Set default zxcvbn threshold to 10k guesses.
See the discussion in the revised docs for background and motivation,
and an explanation of why this value.
2017-10-08 15:48:44 -07:00
Greg Price a116303604 passwords: Express the quality threshold as guesses required.
The original "quality score" was invented purely for populating
our password-strength progress bar, and isn't expressed in terms
that are particularly meaningful.  For configuration and the core
accept/reject logic, it's better to use units that are readily
understood.  Switch to those.

I considered using "bits of entropy", defined loosely as the log
of this number, but both the zxcvbn paper and the linked CACM
article (which I recommend!) are written in terms of the number
of guesses.  And reading (most of) those two papers made me
less happy about referring to "entropy" in our terminology.
I already knew that notion was a little fuzzy if looked at
too closely, and I gained a better appreciation of how it's
contributed to confusion in discussing password policies and
to adoption of perverse policies that favor "Password1!" over
"derived unusual ravioli raft".  So, "guesses" it is.

And although the log is handy for some analysis purposes
(certainly for a graph like those in the zxcvbn paper), it adds
a layer of abstraction, and I think makes it harder to think
clearly about attacks, especially in the online setting.  So
just use the actual number, and if someone wants to set a
gigantic value, they will have the pleasure of seeing just
how many digits are involved.

(Thanks to @YJDave for a prototype that the code changes in this
 commit are based on.)
2017-10-08 15:48:44 -07:00
Greg Price aa4104a5af logging: Add option to show the PID in each log message. 2017-10-06 19:21:40 -07:00
Tim Abbott 0c44310009 tests: Don't use caching session engine in tests.
Because we run tests in parallel, the caching session engine can lead
to nondeterministic failures.
2017-10-06 12:22:20 -07:00
Tim Abbott 5ca7d15e39 urls: Fix name of /plans page.
This was incorrectly duplicating the name of another page.
2017-10-05 20:49:35 -07:00
Tim Abbott b61d667f68 legacy_urls: Remove unnecessary imports. 2017-10-05 15:44:45 -07:00
Alena Volkova 21bc0e845c urls: Delete the json/tutorial_send_message endpoint.
This endpoint is part of the old tutorial, which we've removed, and
has some security downsides as well.

This includes a minor refactoring of the tests.
2017-10-05 15:41:57 -07:00
Vishnu Ks 119157b205 docs: Update email testing section to include EmailLogBackEnd.
Rewritten by tabbott for extra clarity.
2017-10-04 14:44:58 -07:00
Vishnu Ks eef72a98e4 backends: Create custom email backend EmailLogBackEnd.
Create a new custom email backend which would automatically
logs the emails that are send in the dev environment as
well as print a friendly message in console to visit /emails
for accessing all the emails that are sent in dev environment.
Since django.core.mail.backends.console.EmailBackend is no longer
userd emails would not be printed to the console anymore.
2017-10-04 08:20:29 -07:00
Umair Khan 4ed182ef44 django-auth-ldap: Upgrade to 1.2.15.
In 1.2.15 version of django-auth-ldap, the authenticate() function of
LDAPBackend takes username and password as keyword arguments. This
commit updates the code to match this change.

Fixes #6588
2017-10-03 11:40:26 -07:00
Tim Abbott 7581e0e1c5 settings: Remove remaining references to REALMS_HAVE_SUBDOMAINS. 2017-10-02 16:43:54 -07:00
Tim Abbott 1d72629dc4 subdomains: Hardcode REALMS_HAVE_SUBDOMAINS=True. 2017-10-02 16:42:43 -07:00
Umair Khan 69ccc8ce0e ldap: Show helpful message when realm is None. 2017-09-30 10:18:25 -07:00
Tim Abbott 63bbbba5aa backend: Add support for mobile_flow_otp in social auth.
It turns out that very little code change is required to support
GitHub auth on mobile.  Ideally, this would come with tests, though
the complicated part of the code path is covered by the Google auth
version.  But writing a test for this would take a long time, and I
think it's worth having the feature now, so I'll be doing tests as a
follow-up project.
2017-09-30 09:02:46 -07:00
Tim Abbott 16c4ec3a2e settings: Cleanup AUTH_LDAP_BIND_PASSWORD use of get_secret.
At this point, it's pretty reasonable to remove the old way of storing
this in settings.py directly.
2017-09-29 22:47:09 -07:00
Tim Abbott 3e9da25571 settings: Organizing the first section of DEFAULT_SETTINGS. 2017-09-29 22:44:46 -07:00
Tim Abbott a7e81baed8 settings: Document SEND_LOGIN_EMAILS setting. 2017-09-29 22:39:58 -07:00
Tim Abbott c73c12f0e7 settings: Document privacy policy setting.
Also clean up the text around the TERMS_OF_SERVICE setting a bit.
2017-09-29 22:37:56 -07:00
Tim Abbott f0c8e75f57 settings: Remove obsolete VERBOSE_SUPPORT_OFFERS.
We removed the actual functionality of emails differing depending on
this setting several months ago.
2017-09-29 22:35:12 -07:00
Tim Abbott 0374664fb5 settings: Clean up ZULIP_PATHS conditionals.
This replaces the ugly fixed list of things that don't go into var/log
with a reasonable test based on the actual path.
2017-09-29 22:33:32 -07:00
Tim Abbott fb846c567d settings: Remove STATS_DIR.
The use case was removed in 2013 in
81bb2603eb.
2017-09-29 22:32:20 -07:00
Tim Abbott db97c6f77e settings: Remove obsolete PERSISTENT_QUEUE_FILENAME.
We removed the corresponding pickle-based storage code a long time
ago.
2017-09-29 22:30:30 -07:00
Tim Abbott bf2ee2e686 settings: Cleanup logic for ANALYTICS_LOCK_PATH.
We can use the standard ZULIP_PATHS system for this rather code in
dev_settings.py.
2017-09-29 22:29:59 -07:00
Tim Abbott ef5a4000a9 settings: Clean up social auth settings.
The main change here is moving SOCIAL_AUTH_FIELDS_STORED_IN_SESSION to
be with the other hardcoded settings, since it's not something that
makes sense for a sysadmin to change.  But while we're at it, we also
group the overall social auth settings separately from the
GitHub-specific settings.
2017-09-29 22:23:29 -07:00
Tim Abbott 6f343e6bb0 settings: Document remaining undocumented settings with comments.
There's a good number of TODO comments; we should plan to remove most
of these in follow-up cleanup work.
2017-09-29 22:21:43 -07:00
Tim Abbott 698b2bf545 settings: Document the dev/test settings section. 2017-09-29 21:57:29 -07:00
Tim Abbott f920115f18 settings: Remove configuration for EMAIL_GATEWAY_BOT.
This bot user is handled by INTERNAL_BOTS automatically.
2017-09-29 21:44:20 -07:00
Tim Abbott 2a81060acd settings: Organization optional system bots. 2017-09-29 21:44:20 -07:00
Tim Abbott 2f7f8f1cf0 settings: Clean up EMAIL_GATEWAY_PASSWORD.
We haven't ever recommended that people put EMAIL_GATEWAY_PASSWORD in
their /etc/zulip/settings.py, and so this is probably just
ultra-legacy code.
2017-09-29 21:35:55 -07:00
Tim Abbott 94ba678f5b email_mirror: Clean up EMAIL_GATEWAY_EXAMPLE setting.
This isn't something that a user can ever modify, so it doesn't belong
in DEFAULT_SETTINGS.  While we're at it, we align the appearance of
the email gateway in the docs with whether this setting in the docs
will be valid.
2017-09-29 21:32:26 -07:00
Greg Price e90b37d9a6 settings: Remove GCM key from DEFAULT_SETTINGS.
We handle this with `get_secret`, below.
2017-09-29 21:26:36 -07:00
Greg Price 6718b199e0 settings: Classify DEFAULT_SETTINGS by audience.
This will help identify the settings that need attention: either
to remove, or to document for server admins, or to just add a
comment to explain.

Identified with the following shell "one-liner" (one 313-char line
as I originally ran it; indentation added here for clarity):

perl -lne 'next unless (/^DEFAULT_SETTINGS/../\}\)?$/);
           next unless (/'\''(.*?)'\''/);
           print $1' \
    zproject/settings.py \
 | while read var; do \
     echo -n "$var: "; \
     (grep -lw "$var" zproject/{prod_settings_template,{dev,test}_settings}.py \
        || echo none) \
      | sed s,zproject/,,g \
      | fmt -w1000; \
   done
2017-09-29 21:26:36 -07:00
Greg Price b728a2c2d5 settings: Remove S3 key settings from DEFAULT_SETTINGS.
These are handled below with `get_secret`.
2017-09-29 21:26:36 -07:00
Tim Abbott d3bfc132fb settings: Remove TWITTER_ settings from DEFAULT_SETTINGS.
These are handled via `get_secret` instead.
2017-09-29 10:19:43 -07:00
Tim Abbott 0572d5d4a7 settings: Remove SHOW_OSS_ANNOUNCEMENT.
This was used for the old zulip.com to announce that Zulip was now
open source.
2017-09-29 10:19:43 -07:00
Tim Abbott 43f967bccd settings: Remove obsolete VOYAGER setting. 2017-09-29 10:19:43 -07:00
Tim Abbott b38cc4ecf7 settings: Remove obsolete ZULIP_COM setting.
We'll probably soon way to remove VOYAGER as well.
2017-09-29 10:19:43 -07:00
Tim Abbott 6e3f4446a0 settings: Remove DBX_APNS_* settings.
These were only ever used by Dropbox.
2017-09-29 10:19:43 -07:00
Tim Abbott 640d4c9eea settings: Improve DEFAULT_SETTINGS indentation. 2017-09-29 10:19:43 -07:00
Greg Price 412f6e70d8 logging: Add option to show the module a log message came from. 2017-09-28 18:26:39 -07:00
Greg Price 0163920577 logging: Make our own Formatter class.
This doesn't yet do much, but it gives us a suitable place to
add code to customize how log messages are displayed, beyond what
a format string passed to the default formatter can do.
2017-09-28 18:26:39 -07:00
Greg Price e0a5e6fade logging: Explain quirks of Python logging config, and apply a style.
This should make it a little easier to understand our logging config
and make changes to it with confidence.

Many of these items that are now redundant used to be required when we
were setting disable_existing_loggers to True (before 500d81bf2), in
order to exempt those loggers from being cleared out.  Now they're not.

One bit of test code needed a tweak to how it got its hands on the
AdminZulipHandler instance; it can do it from the list on the root
logger just as well as on the `django` logger.
2017-09-28 18:26:39 -07:00
Greg Price f129dc4f72 LDAP: Restore an except clause and add test to cover it.
Most of the paths leading through this except clause were cut in
73e8bba37 "ldap auth: Reassure django_auth_ldap".  The remaining one
had no test coverage -- the case that leads to it had a narrow unit
test, but no test had the exception actually propagate here.  As a
result, the clause was mistakenly cut, in commit
8d7f961a6 "LDAP: Remove now-impossible except clause.", which could
lead to an uncaught exception in production.

Restore the except clause, and add a test for it.
2017-09-28 18:26:39 -07:00
rht f01b629bf9 zproject: Remove absolute_import. 2017-09-27 20:20:07 -07:00
rht b937e1fede Remove the rest of `import six` outside zerver.lib.
We no longer need the six module, now that we're Python 3 only.
2017-09-27 17:09:15 -07:00
Tim Abbott 8d7f961a67 LDAP: Remove now-impossible except clause.
Since we made ZulipLDAPException a subclass of
_LDAPUser.AuthenticationFailed, the django-auth-ldap library already
handles catching it and returning None.

This fixes missing test coverage in this function introduced by
73e8bba379.
2017-09-26 21:33:50 -07:00
Greg Price 315f5b393b logging: Quiet pika.adapters.
This was giving a couple of lines of logs on every normal,
successful connection -- clearly a job for DEBUG, but emitted
on INFO.  Quiet it down.

Fixes #6674.
2017-09-26 21:00:52 -07:00
Greg Price 9e2b9893f9 logging: Organize logger configs for easier reference.
This is a pure reordering.
2017-09-26 21:00:41 -07:00
Greg Price 73e8bba379 ldap auth: Reassure django_auth_ldap our auth-failed exceptions are normal.
The main `authenticate` method in the django_auth_ldap package logs a message
at `exception` level if it passes through an exception it wasn't expecting.
Sensible practice, but we'd been passing through just such an exception for
any kind of routine authentication failure.  After we recently stopped suppressing
an arbitrary subset of loggers with `disable_existing_loggers`, these started
showing up noisily, including in tests.

So, make our exceptions expected.  Just like our own code, the upstream code
raises exceptions of a particular type for routine auth failures, and catches
them and just returns None.  We make our type derive from that one, so as to
just piggyback on that behavior.

Fixes an issue reported in a comment to #6674.
2017-09-26 21:00:41 -07:00
Vishnu Ks 7fd4a71b7f test_emails: Generate emails instead of hardcoding.
Tweaked by tabbott to use require_GET.

Fixes #6344.
2017-09-26 15:02:47 -07:00
Vishnu Ks f58c87917e emails: Log emails that are queued or sent in dev environment.
Tweaked by tabbott to add some comments and clarify the code.
2017-09-26 15:02:47 -07:00
Alena Volkova abe4196232 urls: Change the method for adding alert words from PUT to POST. 2017-09-26 14:00:51 -07:00
Alena Volkova ca687e01d7 urls: Remove the old POST endpoint for alert words. 2017-09-26 14:00:51 -07:00
hollywoodno ee7d4808fe emails: Add log for outgoing emails.
Fixes #5900.
2017-09-25 15:37:09 -07:00
Greg Price 500d81bf2c logging: Stop forcing pre-Python 2.5 legacy config behavior.
The `disable_existing_loggers` option to the `logging.config` module
turns on a rather complicated behavior of disabling some, but not all,
loggers that might have been already configured when the call to
`logging.config.dictConfig` or `logging.config.fileConfig` is made:

> This behaviour is to disable any existing loggers unless they or
> their ancestors are explicitly named in the logging configuration.
  (https://docs.python.org/3/library/logging.config)

Turns out the only reason this is there is as a compatibility hack to
match the behavior of Python 2.4 and below.  See the thread where the
new behavior was introduced: https://bugs.python.org/issue3136

Just as the author of the new behavior explains in that thread from
2008, the legacy behavior forces all logging configuration to be
awkwardly centralized in one place.  That makes the code harder to
read, and it perennially causes confusion when a perfectly
normal-looking `logging.getLogger` call at the top level of one module
mysteriously has no effect, while that in another module works fine,
under the influence of the details of what gets imported when.

So, switch to the shiny new behavior of Python 2.5.  Here LOGGING is a
Django setting which just becomes an argument to logging.config.dictConfig.

This may cause a few of the logfiles in ZULIP_PATHS to become active
that have been dormant for a long time.
2017-09-25 15:06:30 -07:00
Tim Abbott a2243378ea settings: Set REALMS_HAVE_SUBDOMAINS by default.
This change means that almost every Zulip server out there will now be
using subdomains for every realm.  There are a few complications noted
in the release notes.
2017-09-25 06:52:32 -07:00
Eeshan Garg 390a1fec92 zulip_bots: Generate static files during provisioning.
This commit implements support for copying over static files
for all bots in the zulip_bots package to
static/generated/bots/ during provisioning. This directory
isn't tracked by Git. This allows us to have access to files
stored in an arbitrary zulip_bots package directory somewhere
on the system. For now, logo.* and doc.md files are copied over.

This commit should act as a starting point for extending our
macro-based Markdown framework to our bots/API packages'
documentation and eventually rendering these static files
alongside our webhooks' documentation.
2017-09-22 15:14:42 -07:00
Tim Abbott c5cfcd7844 settings: Move the templating configuration further down.
The motivation for this is that we'll want to use the STATIC_ROOT
variable in this code in the upcoming commits.

While we're at it, we give it a proper section in the file.
2017-09-22 15:12:13 -07:00
Juliana Bacelar 928dd06cc8 linter: Add lint rule banning 'import os.path' 2017-09-22 10:32:21 -07:00
Vishnu Ks b4fedaa765 backend: Add support for multiuse user invite link. 2017-09-22 07:56:53 -07:00
julianasobreira df2d448f7b python: Add lint rule banning 'from os.path import'.
This enforces our use of a consistent style in how we access Python
modules; "from os.path import dirname" is a particularly popular
abbreviation inconsistent with our style, and so it deserves a lint
rule.

Commit message and error text tweaked by tabbott.

Fixes #6543.
2017-09-22 04:55:38 -07:00
Tim Abbott b4aa1336e7 logging: Avoid logging certain common 404 errors to error logs.
This should help keep /var/log/zulip/errors.log relatively clean on
production Zulip servers.
2017-09-16 08:07:06 -07:00
Tim Abbott 0d5badb71a urls: Remove robots.txt blocking Zulip serves from search engines.
This was added back in 2012 and seems to have been accidentally not
updated since then.
2017-09-16 01:59:40 -07:00
Supermanu 5f41f3c3cb backends.py: Expose backends that require email usernames 2017-09-15 10:29:02 -07:00
Supermanu 28beddfd76 backends.py: Enable auth with any ldap attributes as username.
This commit enables user to authenticate with any attribute set in
AUTH_LDAP_USER_SEARCH given that LDAP_EMAIL_ATTR is set to an email
attributes in the ldap server. Thus email and username can be
completely unrelated.

With some tweaks by tabbott to squash in the documentation and make it
work on older servers.
2017-09-15 10:28:41 -07:00
Robert Hönig 25742c1199 Capitalize s in Terms of service.
A few strings used `Terms of service`
instead of `Terms of Service`. This change
makes the latter form consistent over the repo.
2017-09-06 09:49:19 -07:00
Brock Whittaker 2140a4aa01 landing: Add /plans/ describing ways to use Zulip.
Note from tabbott: This isn't yet linked to and will need to go
through significantly more iteration, but it's a start.
2017-08-30 07:56:22 -07:00
Steve Howell 0501570cd1 Remove POST-based API for setting topic mutes. 2017-08-29 16:53:38 -04:00
Tim Abbott 50f5560bd1 accounts: Standardize URL for find_account.
This changes it to match the /accounts/ URL style for all of our other
auth code path endpoints.
2017-08-28 14:36:59 -07:00
Tim Abbott ac0d90e533 portico: Rename 'find_my_team' to 'find_account'. 2017-08-28 14:29:29 -07:00
Tim Abbott a0a1fe1512 settings: Rename SERVER_URI to ROOT_DOMAIN_URI.
This should be a lot less confusing.

See #6013 for discussion.
2017-08-28 14:09:28 -07:00
Tim Abbott e092f1afff logging: Fix soft_deactivation log declaration.
Apparently, the soft deactivation log was incorrectly grabbing the
root logger, and thus screwing up where everything got logged.
2017-08-27 18:30:52 -07:00
Tim Abbott b8e7369dee mypy: Remove type: ignores not needed in Python 3. 2017-08-25 11:04:20 -07:00
Umair Khan b473ff54ce Show realm deactivation notice for login/register.
Fixes #6247
2017-08-25 00:09:06 -07:00
Tim Abbott 60105fe38a registration: Remove accounts_home_with_realm_str.
This was the old mechanism for logging into a specific realm on a
server with more than one not using REALMS_HAVE_SUBDOMAINS.
2017-08-24 20:44:52 -07:00
Tim Abbott 0148338a05 casper: Always use REALMS_HAVE_SUBDOMAINS.
This is preparation for setting REALMS_HAVE_SUBDOMAINS to be
permanently True.
2017-08-24 20:44:37 -07:00
Tim Abbott 3d4893d906 settings: Rename SUBDOMAINS_HOMEPAGE to ROOT_DOMAIN_LANDING_PAGE.
This new setting name is a lot more readable.
2017-08-24 19:32:16 -07:00
Tim Abbott be0b523b72 settings: Set STAGING to False in DEFAULT_SETTINGS.
This makes it possible to use this for features like managing the
soft-deactivation cron job running in only one place.
2017-08-23 00:36:06 -07:00
Brock Whittaker d07281fa66 rename: Change list_rendering.js => list_render.js.
The method itself is called `list_render`, so the file should
reflect that name.
2017-08-22 17:37:36 -07:00
Tim Abbott 58edf75962 api: Allow incoming webhook bots to use the send_message API. 2017-08-22 10:06:34 -07:00
Steve Howell ac80d1df0a Add /config-error/smtp endpoint for SMTP errors. 2017-08-21 15:39:26 -07:00
Tim Abbott 134c02e14e settings: Enable GitHub and Google auth by default in dev.
We now show the GitHub/Google auth buttons by default, and just have a
reasonable error message for anyone who clicks them.

Fixes #3651.
2017-08-16 10:05:19 -07:00
Vaida Plankyte 52046d537a auth.py: Add config_error page for misconfigured github/google auth.
Significantly modified by tabbott to use a better system, pass tests,
and clean up the content.
2017-08-16 10:05:19 -07:00
Tim Abbott 71b3245905 test_settings: Include fake keys for Google/GitHub.
This is necessary for tests the to pass with the new config_error code
path, which we don't want to trigger except on specific tests for
those features.
2017-08-16 09:42:37 -07:00
Aditya Bansal 34d30706da soft-deactivation: Log users which were soft deactivate/reactivated. 2017-08-15 22:05:19 -07:00
Tim Abbott 4725afe99a settings: Move NOREPLY_EMAIL_ADDRESS to DEFAULT_SETTINGS.
Since we're auto-detecting the value anyway, there's no reason it
can't be moved to DEFAULT_SETTINGS.

This lets us remove some clutter from the installation documentation.
2017-08-15 17:21:40 -07:00
Tim Abbott d9c8c4db09 settings: Clean up documentation for configuring outgoing email.
Since we now discuss Gmail in the prod-email instructions, we just
link to that here.
2017-08-15 17:21:40 -07:00
Brock Whittaker 83184d5c60 /about/: Merge with /authors, redesign page.
This adds the authors to the Zulip repository on GitHub from
/authors/ along with re-styling the page to fit the same
aesthetic as /for/open-source/ and other product-pages.
2017-08-15 16:09:01 -07:00
Steve Howell 89f9017686 Create new endpoints for marking streams/topics as read.
The new endpoints are:
    /json/mark_stream_as_read: takes stream name
    /json/mark_topic_as_read: takes stream name, topic name

The /json/flags endpoint no longer allows streams or topics
to be passed in as parameters.
2017-08-15 10:17:29 -07:00
Greg Price 12fa1d406f Update iOS app id to reflect the current app.
With this change, we get as far as printing the message
"APNS: Sending apple push notification to devices" to the
log when a recent TestFlight build of the app is due for
a notification, and then don't hit an exception.  But
on the other hand I still don't get an actual notification
on my phone, so there's still some debugging to do.
2017-08-14 17:41:57 -07:00
Steve Howell 57f17ae543 Extract top_left_corner.js.
Here are the functions in top_left_corner:

    get_global_filter_li: pure code move
    update_count_in_dom: simplifed copy of similar function in stream_list.js
    update_dom_with_unread_counts: pure code move, split out from function
       of same name in stream_list.js
    delselect_top_left_corner_items: pure code move
    handle_narrow_activated: pure code move + rename
    handle_narrow_deactivated: pure code move, split out from from function
       of smae name in stream_list.js
2017-08-14 13:03:57 -07:00
rht ea5fef45b5 mypy: Annotate settings.py. 2017-08-14 09:47:28 -07:00
Jack Zhang fc6c2b321b portico: Reimplement /apps routes using pushState. 2017-08-10 10:21:53 -07:00
Jack Zhang 2b6c84a01e urls: Add wildcard to IntegrationView routes after /integrations. 2017-08-10 10:21:53 -07:00
Jack Zhang 63ad7b6769 Rename /integrations/doc API endpoint as /integrations/doc-html. 2017-08-10 10:21:53 -07:00
Greg Price 9cb9e0d687 Revert "mypy: Ensure realm_subdomain is not None in LDAP authenticate()."
I was too hasty in pushing this -- it looks right logically, but it
breaks a test.  May not be hard to fix forward, but reverting now to
unbreak the build in master.

This reverts commit 02acd467b4.
2017-08-09 20:12:27 -07:00
neiljp (Neil Pilgrim) 02acd467b4 mypy: Ensure realm_subdomain is not None in LDAP authenticate(). 2017-08-09 18:04:21 -07:00
Vaida Plankyte cd560a80f2 settings.py: Add GOOGLE_OAUTH2_CLIENT_ID to DEFAULT_SETTINGS.
Allows the command-line help text to appear for Google
authentication, rather than a traceback.
2017-08-07 18:38:06 -07:00
Steve Howell 541156792e Add /mark_all_as_read endpoint.
This change simplifies how we mark all messages as read.  It also
speeds up the backend by taking advantage of our partial index
for unread messages.  We also use a new statsd indicator.
2017-08-04 14:10:46 -07:00
Rishi Gupta 082c3a3c93 portico: Add for-working-groups-and-communities.md. 2017-08-02 00:19:34 -07:00
Rishi Gupta 94f7a5704f portico: Add for/companies. 2017-08-02 00:19:34 -07:00
James Rowan db4a8227ee realm creation: Move DEFAULT_NEW_REALM_STREAMS out of settings. 2017-08-01 22:38:22 -07:00
Steve Howell 3f06f28ad7 sending messages: Extract sent_messages.js.
This commit extract send_messages.js to clean up code related
to the following things:

    * sending data to /json/report_send_time
    * restarting the event loop if events don't arrive on time

The code related to /json/report changes the following ways:

    * We track the state almost completely in the new
      send_messages.js module, with other modules just
      making one-line calls.

    * We no longer send "displayed" times to the servers, since
      we were kind of lying about them anyway.

    * We now explicitly track the state of each single sent
      message in its own object.

    * We now look up data related to the messages by local_id,
      instead of message_id.  The problem with message_id was
      that is was mutable.  Now we use local_id, and we extend
      the local_id concept to messages that don't get rendered
      client side.  We no longer need to react to the
      'message_id_changed' event to change our hash key.

    * The code used to live in many places:
        * various big chunks were scattered among compose.js,
          and those were all moved or reduced to one-line
          calls into the new module
        * echo.js continues to make basically one-line calls,
          but it no longer calls compose.report_as_received(),
          nor does it set the "start" time.
        * message_util.js used to report received events, but
          only when they finally got drawn in the home view;
          this code is gone now

The code related to restarting the event loop if events don't arrive
changes as follows:

    * The timer now gets set up from within
      send_messages.message_state.report_server_ack,
      where we can easily inspect the current state of the
      possibly-still-in-flight message.

    * The code to confirm that an event was received happens now
      in server_events.js, rather than later, so that we don't
      falsely blame the event loop  for a downstream bug.  (Plus
      it's easier to just do it one place.)

This change removes a fair amount of code from our node tests.  Some
of the removal is good stuff related to us completing killing off
unnecessary code.  Other removals are more expediency-driven, and
we should make another sweep at ramping up our coverage on compose.js,
with possibly a little more mocking of the new `send_messages` code
layer, since it's now abstracted better.

There is also some minor cleanup to echo.resend_message() in this
commit.

See #5968 for a detailed breakdown of the changes.
2017-08-01 08:58:56 -07:00
Greg Price 2d534e7397 Remove local_settings.py, as it grows stale with disuse.
This file hasn't reflected the actual configuration of any live
installation for some time, nor been part of any tests or other
mechanism to regularly validate it, so it's naturally fallen
behind as we make changes to the set of settings and typically
don't update this file accordingly.  Just remove it; all the
documentation functions it serves are already served just as
well by prod_settings_template.py and its ample comments.
2017-07-31 21:27:32 -07:00
vbNETonIce 4a5f82bc71 settings: Add default for INLINE_URL_EMBED_PREVIEW.
Tweaked by tabbott to improve the documentation around this setting
and its INLINE_IMAGE_PREVIEW cousin.
2017-07-31 21:11:12 -07:00
Brock Whittaker 8445f886d7 Add ability to pan and zoom lightbox images.
This adds the ability to pan and zoom lightbox images because they
are now converted to <canvas> elements.
2017-07-31 17:03:31 -07:00
Tim Abbott 41f37d0a6f jinja2: Use function interface for render_markdown_path.
Apparently, the filters implementation was doing some sort of strange
caching, where you would need to restart the server in order to
refresh for changes to the markdown content.

We fix this by switching to just calling the render_markdown_path
function from Jinja2.

Fixes #5974.
2017-07-31 14:54:54 -07:00
Tim Abbott a9fa1a5527 api: Migrate /json/messages_in_narrow off legacy API.
This completes the major endpoint migrations to eliminate legacy API
endpoints from Zulip.

There's a few other things that will happen naturally, so I believe
this fixes #611.
2017-07-31 13:08:06 -07:00
Tim Abbott 8003428845 api: Document remaining legacy endpoints. 2017-07-31 13:08:06 -07:00
Tim Abbott ed10ce70d4 invites: Migrate to non-legacy API format.
This has the side effect of making it possible for a bot user to
invite users with its API key, which seems useful.
2017-07-31 13:08:06 -07:00
Tim Abbott 6a50e13156 uploads: Remove legacy /json/upload_file endpoint.
This migrates Zulip to use the equivalent API endpoint that has been
present for a while.
2017-07-31 13:08:06 -07:00
Tim Abbott 1e5aee054b settings: Migrate main settings-change code to API.
This was one of the few major remaining endpoints that were still on
the old-style legacy API.
2017-07-31 13:08:06 -07:00
Jack Zhang c242df042f hotspots: Style hotspots. 2017-07-28 16:34:15 -07:00
vbNETonIce ea72432dca rename in comment: scripts/restart-zulip to scripts/restart-server
found this when trying to follow this comment and the script didn't exist anymore
2017-07-28 10:29:17 -07:00
Vaida Plankyte 26f5f77524 prod_settings_template.py: Update GitHub/Google auth setup docs. 2017-07-27 17:35:14 -07:00
Tim Abbott 53e4d8562b lint: Add a lint check for bare `type: ignore`s. 2017-07-27 16:31:55 -07:00
Steve Howell bc0761b22b Extract topic_data.js.
This new module tracks the recent topic names for any given
stream.

The code was pulled over almost verbatim from stream_data.js,
with minor renames to the function names.

We introduced a minor one-line function called stream_has_topics.
2017-07-27 14:26:22 -07:00
Rishi Gupta 6439603974 prod_settings_template: Add quotes around push notification url. 2017-07-25 17:42:33 -07:00
Greg Price e18baff32c JsonableError: Rename message from `error` to `msg`.
The whole thing is an error, so "message" is a more apt word for the
error message specifically.  We abbreviate that as `msg` in the actual
HTTP responses and in the signatures of `json_error` and friends, so
do the same here.
2017-07-24 16:41:22 -07:00
Rishi Gupta 9782911aca portico: Add initial why zulip page. 2017-07-19 11:53:06 -07:00
Pweaver (Paul Weaver) 01b0ec69d6 Move the webpack stats files to var/ and deploy root.
static/ serves static files which get copied around per deploy. Since
the webpack stats files need a consistent name and change per deploy,
they can't live in static/.

This fixes a bug that preventing downgrading a Zulip server to an old
version.
2017-07-18 15:02:58 -07:00
Rishi Gupta a249822ff5 confirmation: Add validity_in_days to _properties.
Also renames settings.EMAIL_CONFIRMATION_DAYS to
CONFIRMATION_LINK_DEFAULT_VALIDITY_DAYS, and adds a new setting for
invitation links.
2017-07-17 23:18:47 -07:00
Steve Howell 475eb21a5e Revert commits related to client_message_id.
I pushed a bunch of commits that attempted to introduce
the concept of `client_message_id` into our server, as
part of cleaning up our codepaths related to messages you
sent (both for the locally echoed case and for the host
case).

When we deployed this, we had some strange failures involving
double-echoed messages and issues advancing the pointer that appeared
related to #5779.  We didn't get to the bottom of exactly why the PR
caused havoc, but I decided there was a cleaner approach, anyway.
2017-07-14 12:13:35 -07:00
Steve Howell f6d670ae3d Extract sent_messages.js.
This is mostly straightforward moving of code out of compose.js.

The code that was moved currently supports sending time
reports for sent messages, but we intend to grow out the new
module to track more state about sent messages.

The following function names in this commit are new, but their
code was basically pulled over verbatim:

    process_success (was process_send_time)
    set_timer_for_restarting_event_loop
    clear
    initialize

All the code in the new module is covered by previous tests that
had been written for compose.js.  This commit only modifies
a few things to keep those tests.

The new module has 100% node coverage, so we updated `enforce_fully_covered`.
2017-07-13 23:42:27 -04:00
Jack Zhang 6e2f90c8c9 Improve /integrations page load time.
Prior to this commit, 7 megabytes of images (through 253 individual requests)
were heavily slowing down the initial load. With this commit, we load only the
logos (60 or so images).

Documentation and images for the individual integration sub-pages is requested
separately using the /integrations/doc/ endpoint, which returns HTML.
2017-07-13 20:37:09 -07:00
Brock Whittaker 1cf7ee966b help: Redesign /help/ pages to be a single page app.
This redesigns the /help/ page sets to be a single page app that uses
history.pushState to work the same as the old app.

The big new feature is that now we have the index in a nicely designed
left sidebar.
2017-07-13 15:43:44 -07:00
Umair Khan f94e8530ee Django 1.11: Remove jinja2 backend.
Django 1.11 adds the ability to pass context processors in Jinja2
backend. Django also sends template_rendered signal in tests.
These two issues were the reason why we added Jinja2 backend, but
after upgrading to Django 1.11 we can remove it.

We still need jinja2/__init__.py, which modifies the environment,
and jinja2/compressors.py, which adds minify_js compressor.
2017-07-12 17:53:25 -07:00
Aditya Bansal b9bffaa446 pep8: Add compliance with rule E261 to prod_settings_template.py. 2017-07-11 11:55:01 -07:00
Aditya Bansal 5f87a0f8f8 pep8: Add compliance with rule E261 zproject/settings.py. 2017-07-11 11:53:33 -07:00
Aditya Bansal 7c6ad4a770 pep8: Add compliance with rule E261 zproject/dev_settings.py. 2017-07-11 11:53:33 -07:00
Umair Khan 816ead737c jinja2: Support only Django version >= 1.11.
Django started supporting context processors from version 1.11; as
a result of this, we can get rid of some of the code which is now
being taken care of by Django.
2017-07-10 11:32:10 -07:00
Rishi Gupta 0f46fd86dd urls: Rename arguments to accounts/unsubscribe.
type -> email_type to match future work on ScheduledJob.

token -> confirmation_key to match what the other confirmation views urls
call this argument.
2017-07-07 18:56:14 -07:00
Vaida d5517bae36 Delete the old zulip.com "referrals" system.
This system hasn't been in active use for several years, and had some 
problems with it's design.  So it makes sense to just remove it to declutter
the codebase.

Fixes #5655.
2017-07-07 14:59:18 -07:00
Vishnu Ks 9067890120 urls: Use POST for zerver.views.user_settings.set_avatar_backend.
The set_avatar_backend is changed from PUT to POST becuase this
is not idempotent.
2017-07-07 10:31:43 -07:00
Vishnu Ks df5df76961 urls: Use POST for zerver.views.realm_emoji.upload_emoji.
The upload_emoji endpoint is changed from PUT to POST because
this endpoint is not idempotent.
2017-07-07 10:31:43 -07:00
Vishnu Ks cf2dec539f urls: Use POST for zerver.views.realm_icon.upload_icon.
This upload_icon endpoint is changed from PUT to POST becuase
this endpoint is not idempotent.
2017-07-07 10:31:43 -07:00
Rishi Gupta 07a3bb4d35 confirmation: Remove get_link_validity_in_days.
This commit removes the ability to configure different validity durations
for different types of confirmation links. I don't think the extra
configurability was worth the extra complexity, either for the user trying
to understand the settings, or for the developer trying to understand the
code.

The commit replaces all confirmation validity duration settings with a
single setting, settings.EMAIL_CONFIRMATION_DAYS.

The only setting it removes is settings.EMAIL_CHANGE_CONFIRMATION_DAYS,
which was introduced in 5bf83f9 and never advertised in prod_settings.py.
2017-07-07 07:46:31 -04:00
James Rowan 368bd66d8b emails: Refactor send_email functions to take both a sender name and address.
This will allow for customized senders for emails, e.g. 'Zulip Digest' for
digest emails and 'Zulip Missed Messages' for missed message emails.

Also:
* Converts the sender name to always be "Zulip", if the from_email used to
  be settings.NOREPLY_EMAIL_ADDRESS or settings.ZULIP_ADMINISTRATOR.

* Changes the default value of settings.NOREPLY_EMAIL_ADDRESS in the
  prod_setting_template to no longer have a display name. The only use of
  that display name was in the email pathway.
2017-07-04 14:25:01 -07:00
Rishi Gupta dd3dd26f15 prod_settings_template: Update description for FEEDBACK_EMAIL. 2017-07-04 14:25:01 -07:00
Cory Lynch 802ea9abf5 Refactor to delete mousewheel.js.
This old third party library added support
for a "mousewheel" event to detect scrolling.
However, it is not compatible with jQuery 3
and is obsolete now that there is a standard
"wheel" event that accomplishes the same thing.
2017-07-03 11:04:20 -04:00
Pweaver (Paul Weaver) 2c10e7c0db Change from and noreply emails to use EXTERNAL_HOST domain.
This eliminates another mandatory setting from what one needs to
configure to setup a Zulip server.
2017-07-01 08:09:25 -07:00
Rishi Gupta 2012913cc1 prod_settings_template: Update description of NOREPLY_EMAIL_ADDRESS. 2017-06-29 17:54:33 -07:00
Rishi Gupta 85d38bd17b emails: Remove DEFAULT_FROM_EMAIL from prod_settings_template. 2017-06-29 17:54:33 -07:00
Rishi Gupta 577e201df0 emails: Replace DEFAULT_FROM_EMAIL with ZULIP_ADMINISTRATOR.
We're getting rid of DEFAULT_FROM_EMAIL from prod_settings_template, since
we only really need one of {DEFAULT_FROM_EMAIL, ZULIP_ADMINISTRATOR}.
2017-06-29 17:54:33 -07:00
Vishnu Ks 9b7585a436 bots: Make welcome-bot cross realm. 2017-06-25 11:23:45 -04:00
Umair Khan a66b7b7083 Remove Django Guardian dependency.
Guardian adds functionality on top of Django auth system to set
per object permissions. Its problem is that it is has poor performance.
So we decided to remove it in release 1.4.0, but we still kept the
option to revert back to an older version which used Guardian.
See commit 49799440a4 for more details.

This commit is the final piece in the string of commits which move
us towards completely removing guardian from our codebase. The way
we do it as follows:

If you are upgrading from a version <= 1.3.10, you first need to
upgrade to 1.4.x (we recommend 1.4.2). The reason is that we
deprecated Guardian in this version. Once you have upgraded to
1.4.x we can be sure that your Zulip installation doesn't depend
on Guardian and all the data has been successfully migrated away from
Guardian. The second step is to upgrade to latest release which will
not include any reference to Guardian in the codebase. After this
commit migrating directly to the latest release will not work because
in that case Guardian data will not migrate.

The backward incompatible change that this introduces is that
we have squashed all the migrations till version 1.4.0. This was
necessary to remove Guardian because it was needed by the reverse
migration. These migrations were from 0001 to 0028.

Fixes #5420
2017-06-23 08:40:57 -04:00
Greg Price c9e35c459a landing: Write up why open source projects should use Zulip
This isn't very slick, but it should get the main points down,
and it's past time we got something like this up.  Definitely
needs in the future another pass at the text, and also some images
(screenshots, etc.) and styling.
2017-06-22 16:26:50 -07:00
Harshit Bansal c0e4f24a25 webpack: Migrate `common.js` bundle to webpack. 2017-06-22 19:06:32 -04:00
Vaida Plankyte 7577a80fd9 settings.py: Change jinja2 blocks to be stripped of whitespace. 2017-06-22 14:19:33 -04:00
Umair Khan a4afca7b73 ldap: Don't authenticate if realm is None.
Fixes #5431
2017-06-21 10:14:34 -04:00
Harshit Bansal c50b3672c2 webpack: Migrate `signup.js` bundle to webpack static asset pipeline. 2017-06-20 09:45:26 -04:00
Harshit Bansal 3953734a1c webpack: Migrate `stats.js` bundle to webpack static asset pipeline. 2017-06-20 09:45:26 -04:00
Cory Lynch 78de70688e dev: Add URL to preview error 404/5xx pages. 2017-06-14 21:18:45 -07:00
Cory Lynch cb4dadfdcd tools: Create /devtools page for useful dev URLs.
Fixes #5179.
2017-06-14 20:17:56 -07:00
Umair Khan 95fc16d90d Django 1.11: MIDDLEWARE_CLASSES setting is deprecated.
Django provides MiddlewareMixin to upgrade old-style middlewares. See
https://docs.djangoproject.com/en/1.11/topics/http/middleware/#upgrading-middleware
2017-06-13 15:04:04 -07:00
Umair Khan c25e9ad193 Django 1.11: Template now accepts backend in __init__.
This makes our `zproject.jinja2.backend.Template` compatible with
Jinja2. After this change we don't need to override __init__ function
in Template class.

The only reason we now need to create our own Template class is that
we need to send template_rendered signals.

We need our own Jinja2 class because we need to maintain backward
compatibility with Django 1.10 and we need inject `debug` parameter.
2017-06-13 15:04:04 -07:00
Pweaver (Paul Weaver) 28874cf26f webpack: Add --minify option to run-dev.py for to test minification. 2017-06-13 12:45:40 -07:00
Mehanig 1e5ce918e3 webpack: Move app_debug.js to webpack bundle. 2017-06-13 12:45:40 -07:00
Mehanig 04626acd5a webpack: Move zxcvbn.js to webpack bundle. 2017-06-13 12:45:39 -07:00
Mehanig ae29d4b8d2 webpack: Move landing-page.js to webpack bundle. 2017-06-13 12:36:55 -07:00
Mehanig 2c6f2fb93a webpack: Move activity js to webpack. 2017-06-13 12:36:51 -07:00
Pweaver (Paul Weaver) 5c07125994 Enable EmailAuthBackend by default to simplify a production install.
This should save a step from the installation process in the common
case.
2017-06-12 21:53:35 -07:00
Rafid Aslam 4f482834b4 deps: Upgrade and move `perfect-scrollbar` from `static/third` to `npm`.
- Remove `perfect-scrollbar` from `static/third` and fetch it from npm.
- Upgrade `perfect-scrollbar` to 0.7.1.
- Bump up the `PROVISION_VERSION` to 5.6.

Changed `wheelSpeed` in "static/js/scroll_bar.js" to 0.5, because when it
20, the scrollbar scrolls very fast.

Changed 'wheelSpeed' in "static/js/emoji_picker.js" from 25 to 0.68
(based on tabbott's testing of scrolling through the emoji list).

Part of #1709.
2017-06-11 10:15:48 -07:00
Cory Lynch 6219128d87 dev: Add /emails/ URL for testing email templates.
Fixes #5270.
2017-06-09 16:21:49 -07:00
Cory Lynch 6653e19e3a Add URL for viewing node coverage data on dev server.
Running test-js-with-node --coverage now provides a URL
to view the output data on the web.

Fixes #5177.
2017-06-09 16:17:30 -07:00
Mehanig 8c0e500d6b webpack: Migrate api and katex js files to Webpack bundler.
This also injects the dev webpack sockets to katex, since that's part
of the main web app bundle.
2017-06-09 12:33:52 -07:00
Rishi Gupta 769c5ab105 emails: Send password reset emails through zerver.lib.send_email.
Previously, the password reset email behaved differently from all the other
email Zulip sends.
2017-06-06 23:22:22 -07:00
Rishi Gupta 15e5814c27 emails: Prevent subject lines and plaintext emails from being HTML-escaped.
Adds a new template engine with autoescape=False, and uses the new engine to
render subject lines and plaintext emails.
2017-06-06 23:22:22 -07:00
Rishi Gupta 4d67ce8051 settings.py: Make template engine settings easier to duplicate.
No functional changes.
2017-06-06 23:22:22 -07:00
Greg Price 04b363f3a2 apps: Redirect /apps -> zulipchat.com/apps, except on upstream itself.
This page describes software the user will get from upstream for
their own devices, independent of what's on the server they're
using.  So it should live in a place maintained together with
that other software, rather than be distributed and versioned
with the server.

The use of ZILENCER_ENABLED to tell the difference is rather a hack
but is currently how we do this in the small handful of similar
spots; see #5245.

Fixes #5234.
2017-06-06 11:21:20 -07:00
Aditya Bansal feb663ffb6 pep8: Add compliance with rule E261 backends.py. 2017-06-04 09:18:22 -07:00
Cory Lynch 5d7828096e Split out server_events_dispatch.js from server_events.js. 2017-06-02 16:49:18 -07:00
Tim Abbott 4da9383f2d dev_urls: Add comments documenting sections. 2017-06-02 12:44:32 -07:00